Offload cached pages to different Apache process

Question

On our Drupal 6 and Drupal 7 sites each idle Apache process averages 60-100MB each. When the page gets loaded from cache only about 10MB is really needed, but sometimes a page is not cached yet so the process jumps up to 60-100+MB.

We use prefork, so each apache process stays about 60-100MB in size, when in reality 99.99% of the page loads are cached and the processes could easily be down to 10MB.

Our MaxClients settings could be about 5x greater if there were a way to keep the apache sizes down to 10MB.

Is there a way to have two apache processes available, one with a low maxclients for non-cached pages and one with a high maxclients for cached pages? And also, is there a way to route cached & non-cached requests to different processes?

Answer 1

It is not a direct answer to your question, but play around with setting MaxMemFree and MaxRequestsPerChild in the prefork section of your httpd.conf. MaxMemFree should force Apache to free memory that it doesn't need, and MaxRequestsPerChild will force Apache to fork a new process (and free memory in the process) after servicing a particular number of requests. You may have some performance impacts, but gain memory improvements.

Answer 2

I'd suggest moving to the idea of a dedicated, lightweight, configurable cache daemon, for example Varnish which can run in its own right, taking the load off apache for the majority of cache hits for the site, and can even be moved onto a separate physical server in the future should you still need more power.

Answer 3

Apache would need to know if the cache exists if it where to route the process; it currently doesn't. http://httpd.apache.org/docs/trunk/mod/mod_dbd.html might be one idea if you wish to create your own apache module that will query the cache_page table. Another option is to look at using something like boost http://drupal.org/project/boost which will not start up PHP on page cache hits.