10

I'm curious if anyone has made attempts to "cache" the bootstrap process in Drupal.

Normally, Drupal will run the 7 bootstrap phases on each request, but perhaps on a deployed production system, one could "do away" with some or all of these?

Possible suggestions I have in mind could be

  1. Serializing the bootstrapped state and sticking it into memcache
  2. A script could generate a patch for bootstrap.inc that will hardcode certain information into the file.
  3. I believe David Strauss has tried to keep a bootstrapped Drupal running on libevent.
  4. Other craziness?

What attempts are there, and which are known to be (somewhat) reliable?

12

PHP is a shared nothing architecture. That has its advantages and disadvantages.

One disadvantage is that it's not easy to do something like this. There isn't much of a state that can be stored somewhere.

I did some quick tests and when logged in, then the boostrap seems to take about ~17% of the total time and more than 50% of that is actually loading all the .module and .inc files. That's not something that you can store in memcache. Also, it doesn't seem to matter much if I use memcache or the database cache.

I tried to get some results when having the page cache enabled, but Xhprof doesn't seem to return reliable results then; the whole thing simply seems to be too fast. But even then, the biggest part involves executing init/exit hooks and loading files it seems. I found an interesting issue there: It looks like the User module is seriously slowing down the cached page response because it triggers the registry due to the entity controller in the .module file.

That said, David Strauss showed some experimental work in Copenhagen where he created a memory snapshot after bootstrapping and then returning to that once the page was served. He did use Drupal 6 for that. After looking at the numbers above, I imagine that the performance gains of doing this in Drupal 7 would be quite a bit smaller. One reason for this is that the database connection is lazy loaded (And you can get quite far in the bootstrap when using e.g. Memcache before you need to execute the first query) and there's a lot that is cached.

What's really bad in Drupal 7 is the render layer with these huge arrays and endless recursions and loops. That one pretty much undoes all the performance work that went into Drupal 7. Let's see how it looks in Drupal 8, if Twig makes it into core.

Lastly, about the mentioned advantages. One large advantage is that memory leeks are rather irrelevant because everything is freed after each request. I've seen many Java applications where memory usage constantly increases and need regular restarts.

  • 4
    I so love having you on the site @Berdir ;) – Letharion May 8 '12 at 9:50
  • Alex Bronstein mentioned it during the sprint that it's somewhat slow to include the tpl.php files even with APC it requires a stat -- but Twig compiles to classes so that will be a win on pages like node + many comments. We will see. – user49 May 9 '12 at 10:29
  • I guess you could make a system where, for cached pages, you generate a bunch of rewrite rules and put them in .htaccess, accompanied by html pages to bypass PHP completely. Might not be worth the trouble though: IIS, nginx, logged-in users, ... – Bart May 15 '12 at 12:41
  • 1
    @Bart: You just re-invented Boost: drupal.org/project/boost :) – Berdir May 15 '12 at 18:35
5

Nope, it was David Strauss who was experimenting with kargo-event (now called Kellner) at https://code.launchpad.net/~fourkitchens/pressflow/6-evented but I doubt anything serious came out of it.

Drupal 7 does have a lot of bootstrap already cached, there's a cache_bootstrap bin for that. You can stick it into memcached.

You can go overboard and decrease code loading with moving some / lots of Drupal code into C. Damien and dhthwy created PHP extension at http://drupal.org/project/drupal_php_ext not much is done with it. Or do hiphop. I do not know the current state of hiphop & Drupal 7.

At the end of the day, however, you need to take a good hard look at the engineering costs of, say, getting hiphop work with Drupal 7 (and all the contrib!) and compare it with renting a few more servers. If you can save, say 5% of your servers and you have 100 000 servers, go for it, but are you Facebook? Be careful and economical with optimizations.

  • Thanks, I've updated the question and removed your name from it. :) I realize that for many cases, there are far more efficient ways to deal with performance, I was mostly curious. – Letharion May 9 '12 at 7:14
1

Saw an interesting presentation about doh (Dynamic Object Handler) at the High Performance Drupal meetup. In short he talks about bootstrapping drupal fast by using it. Gets interesting around the 15:30 mark. Autoload on steroids in short with runkit functionally as well. QA at the 33:00 mark.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.