We are having a high traffic Drupal 6.x site (25,000) uniques per day, mostly landing pages.
Everythings works OK, with great performance, until we try to reset the boost cache.
Once we reset it, Drupal tries to regenerate HTML from PHP and MYSQL, and we reach a performance crisis where the site crawls and becomes unusable due to slow performance.
Is there a better solution? I imagine there could be a script which clears cache slowly, resets files one by one, wait(10 seconds), wget the url, and then go to the next url.
More info added (edits):
- The site has about 20,000 nodes.
- We have two Linode 4096 machines: 4 GB RAM, 8 CPU (4x priority), 192 GB Storage, 8 TB Transfer each. Only one of them stores MySQL.
- The reason we do manual purge is because when Boost purges pages by itself, we can get to a point where a lot of pages are not in the cache. This can cause a big performance hit. When the server is too busy, PHP returns WSOD and then Boost caches it! So we get a page, in cache that is actually WSOD!!! Since this is a critical problem, we decided to affectively disable Boost from purging cache on it own and do it On Demand.
- Why clear the boost cache? The CSS and JS files are global. If we change the CSS, or add some JS, then we "drush cc css-js" to load the new changes. Doing that, the CSS/JS files are deleted and we are left with cached HTML pointing to non-existing CSS/JS files. So we get "broken" pages. The solution is to refresh the Boost cache.
Will the advagg module help?
Any other solutions?