Varnish 4.0 vcl for Drupal 7

Question

Drupal7, Varnish 4.0, Ubuntu 12.4

Varnish is installed an working, but returning intermittent 503, I think whenever Varnish goes to the backend. Even though the request returns 503, a page refresh may return 200, HIT from cache. Another refresh may be 200 or 503. I think a subsequent 503 is a HIT-FOR-PASS.

We've opened up the timeouts, as follows:

vcl 4.0;
backend default {
.host = "127.0.0.1";
.port = "8080";
.probe = {
.url = "/ping";
.timeout = 600s;
.interval = 600s;
.window = 5;
.threshold = 2;
}
.first_byte_timeout = 900s; # How long to wait before we receive a first byte from our backend?
.connect_timeout = 500s; # How long to wait for a backend connection?
.between_bytes_timeout = 200s; # How long to wait between bytes received from our backend?
}

So I suppose it's not that Varnish is timing out in a blink of the eye.

Probably the problem is in the vcl; this is what I'm using: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl

Here are fragments from the varnish log for 3 successive calls to the same page: 503, then 200, then 503 again:

First call - returns 503
-   VCL_call       BACKEND_FETCH
-   VCL_return     fetch
-   FetchError     no backend connection
-   Timestamp      Beresp: 1408509130.524000 0.000219 0.000219
-   Timestamp      Error: 1408509130.524013 0.000232 0.000013
-   BerespProtocol HTTP/1.1
-   BerespStatus   503
-   BerespReason   Service Unavailable
-   BerespReason   Backend fetch failed
-   BerespHeader   Date: Wed, 20 Aug 2014 04:32:10 GMT
-   BerespHeader   Server: Varnish
-   VCL_call       BACKEND_ERROR
-   BerespHeader   Content-Type: text/html; charset=utf-8
-   BerespHeader   Retry-After: 5

Page refresh - Returns 200 from the cache VCL_return hash - VCL_call HASH - VCL_return lookup - Hit 2147745802 - VCL_call HIT - VCL_return deliver - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK

Another refresh - returns 503 again.

• VCL_call HASH
• VCL_return lookup
• Debug "XXXX HIT-FOR-PASS"
• HitPass 2147811368
• VCL_call PASS
• VCL_return fetch
• Link bereq 196777 pass
• Timestamp Fetch: 1408509419.224633 0.000666 0.000666
• RespProtocol HTTP/1.1
• RespStatus 503
• RespReason Backend fetch failed

Any clues on getting the vcl correct appreciated.

Answer 1

The VCL you have linked to and your example in the question show you only using one backend. In this case using the probe to do a health check serves no purpose. As per drifteaur's answer disabling probe resolves many 503 issues and as you have no need for it, problem is solved.

However, if you are testing working towards rolling out on multiple backends then it still needs to be fixed(understood).

Looking at the VCL you linked (https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl) it uses many sections which have been taken from https://www.varnish-software.com/ so we can presume (hopefully) that they are correct.

Currently, your probe is checking the response from "/ping" at your host and port, 127.0.0.1:8080, so that is 127.0.0.1:8080/ping .

If your set-up is multiple virtualhosts on one server with varnish sitting infront then using probe like this will not get a response(as this full url has most probably not been configured) and in turn this will lead to the backend being marked as "sick / unhealthy" and a 503 will be produced as Varnish will not even attempt to access the backend.

To confirm that it is your server setup add the following to your probe

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .probe = {
              .request =
              "HEAD / HTTP/1.1"
              "Host: www.EXAMPLE.com"
              "Connection: close";
              }
}

You will need to change www.EXAMPLE.com to a real domain you are using on the server.

If this gives you 200s rather than 503s than it is safe to confirm that using probe as you were with your server set-up was incompatible.

Probes info: https://www.varnish-cache.org/docs/trunk/reference/vcl.html#reference-vcl-probes

Answer 2

I had the same problem (503) with the same example config and Drupal. I resolved it by commenting out the .probe part of the backend configuration. Not sure why that helped though.