Upstream cache clear issues

Question

I'm running into issues with caching on my Drupal 7 site, where it seems that users are receiving stale pages, despite not having "cache pages" or "cache blocks" settings enabled. There is no additional server-side caching enabled. I've cleared the cache via drush cc all, as well as clearing the cache through the UI.

After clearing the cache, and with js aggregation enabled, I've pulled up pages where I receive an old version of the js. With aggregation turned off, I get the new js, as expected. Turn aggregation back on and pull up the page in an incognito window, and I get the old js.

However, even with aggregation turned off, I've also seen in our logs that we're getting "Page not found" messages for expired, aggregated js files. This indicates to me that users are getting served pages with references to these old files, though I'm open to other possibilities.

The headers for our homepage include Cache-Control: no-cache, must-revalidate and an Expires value in the past. The headers for our main.js file include Cache-Control: max-age=1209600, an ETag value, and an Expires value set to 2 weeks from now. I thought that by clearing Drupal caches, there would be an update to the ETag value that would force users to download an updated asset. Our hosting support has suggested that the settings on these assets may be impacting the upstream cache (I'm assuming they mean the cache on the homepage, for example).

So it seems like there is page caching that's happening on some level. Since I don't have server-side caching or page / block caching set through Drupal, it seems that the cache settings on the assets are impacting page cache.

Any input or thoughts on troubleshooting would be appreciated!

UPDATE: In paying more attention to the details of aggregated files and cache clearing, today after clearing the cache, I had an updated, aggregated CSS for a while, and non-aggregated JS with a cache-busting ?examplestring appended. In loading the site a few hours later, I'm being served the old aggregated CSS with the new non-aggregated JS. So it's got to be something wrong with how Drupal includes what it determines to be the newest version of the aggregated files.

Answer 1

I'm not familiar with the term "upstream caching / caches." I'm assuming it has something to do with edge caching... which I definitely suggest you look into; regardless of host service. I've seen many odd occurrences because of edge caching.

Also try any of the following:

check file permissions for the ability for your web server to write the JS/CSS that it's generating. Look in your sites/default/files or site/[site]/files directory for css and js subdirectories. Check the permissions on those two AND delete the .css, .css.gz, .js, and .js.gz files in them. truncate (with discretion - backup your database) all tables with a _cache at the end of them.