Problem with admin_menu and Varnish

Question

I'm a beginner with configuring varnish, so my apologies if this is a stupid question. I am trying to make things work on my ubuntu VM (12.04) with Varnish 3.

I am facing a problem, where after I log in, the admin menu (created by admin_menu) just disappears. Looking at the ajax request, it seems that varnish sends the request to drupal, receives a valid answer and then just refuses to serve it. It just outputs a 503 error instead.

I have captured the varnish logs, but I'm not sure what to make of them. Anyone has any ideas?

    11 ReqStart     c 192.168.1.20 50807 2144510792
   11 RxRequest    c GET
   11 RxURL        c /js/admin_menu/cache/a4270b058219f623d092552467a5cde0
   11 RxProtocol   c HTTP/1.1
   11 RxHeader     c Host: www.giftmonkey.ubuntu
   11 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Firefox/24.0
   11 RxHeader     c Accept: text/plain, */*; q=0.01
   11 RxHeader     c Accept-Language: en-US,en;q=0.5
   11 RxHeader     c Accept-Encoding: gzip, deflate
   11 RxHeader     c X-Requested-With: XMLHttpRequest
   11 RxHeader     c Referer: http://www.giftmonkey.ubuntu/
   11 RxHeader     c Cookie: __utma=94903434.1646672653.1383344251.1383344251.1383390432.2; __utmz=94903434.1383344251.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); has_js=1; pmcwidth=1920; __utmc=94903434; SESS9433506a0f068ea0ad220f58b963eef8=6FfRyBpNYYBlufL7y6AoBa5eqBP
   11 RxHeader     c Connection: keep-alive
   11 VCL_call     c recv pass
   11 VCL_call     c hash
   11 Hash         c /js/admin_menu/cache/a4270b058219f623d092552467a5cde0
   11 Hash         c www.giftmonkey.ubuntu
   11 VCL_return   c hash
   11 VCL_call     c pass pass
   11 Backend      c 17 default default
   11 TTL          c 2144510792 RFC 31536000 -1 -1 1383409657 0 1383409657 0 31536000
   11 VCL_call     c fetch
   11 TTL          c 2144510792 VCL 120 -1 -1 1383409657 -0
   11 VCL_return   c hit_for_pass
   11 ObjProtocol  c HTTP/1.1
   11 ObjResponse  c OK
   11 ObjHeader    c Date: Sat, 02 Nov 2013 16:27:37 GMT
   11 ObjHeader    c Server: Apache/2.2.22 (Ubuntu)
   11 ObjHeader    c X-Powered-By: PHP/5.3.10-1ubuntu3.8
   11 ObjHeader    c Expires: Sun, 02 Nov 2014 11:17:03 +0000
   11 ObjHeader    c Last-Modified: Sat, 02 Nov 2013 11:17:04 +0000
   11 ObjHeader    c Cache-Control: private, max-age=31536000
   11 ObjHeader    c ETag: "1383391024-1"
   11 ObjHeader    c X-Drupal-Cache: HIT
   11 ObjHeader    c Content-Language: ro
   11 ObjHeader    c Vary: Cookie,Accept-Encoding
   11 ObjHeader    c Content-Encoding: gzip
   11 ObjHeader    c Content-Length: 6705
   11 ObjHeader    c Content-Type: text/html; charset=utf-8
   11 FetchError   c Junk after gzip data
   11 Gzip         c u F - 6611 59085 80 80 52818
   11 VCL_call     c error deliver
   11 VCL_call     c deliver deliver
   11 TxProtocol   c HTTP/1.1
   11 TxStatus     c 503
   11 TxResponse   c Service Unavailable
   11 TxHeader     c Server: Varnish
   11 TxHeader     c Content-Type: text/html; charset=utf-8
   11 TxHeader     c Retry-After: 5
   11 TxHeader     c Content-Length: 436
   11 TxHeader     c Accept-Ranges: bytes
   11 TxHeader     c Date: Sat, 02 Nov 2013 16:27:37 GMT
   11 TxHeader     c X-Varnish: 2144510792
   11 TxHeader     c Age: 0
   11 TxHeader     c Via: 1.1 varnish
   11 TxHeader     c Connection: close
   11 TxHeader     c X-Cache: MISS
   11 Length       c 436
   11 ReqEnd       c 2144510792 1383409657.339800358 1383409657.400331020 0.511597395 0.060457230 0.000073433
   11 SessionClose c error
   11 StatSess     c 192.168.1.20 50807 2 1 5 0 5 4 1760 6926

Answer 1

The error see to be on some cache file of drupal's admin_menu. The js aggregated file browser is requesting is being refused by Drupal by 503. You may get some pointers to this issue here https://groups.drupal.org/node/28604

Try clearing all the Drupal caches using "drush cc all" command. Are you using any js aggregation on Drupal side? Try changing to https://drupal.org/project/advagg?

Answer 2

I seem to have resolved this on our system by doing the following:

Navigate to admin/config/administration/admin_menu

Under "Performance" field group - uncheck "Cache menu in client-side browser"

Save

Note: I only have this enabled as such in my dev/stage environments, since that's where 99% of everything 'work' related happens. I suppose it could be 'disabled' in production as well, providing it is only used by admins, and that no 'normal' role, anon, has perms for said item. The caching benefit isn't realized in this type production usage, so it wouldn't matter.