12

On a site of mine drush cc all takes more than 4 mins to run. The site db is a few GBs. However, I cannot see a clear reason as to why it takes too long. What can I do to locate the bottle neck?

  • Check first mysql query log: drupal.stackexchange.com/questions/75629/… – AgA Jan 30 '14 at 17:02
  • Do you have cron running? – mpdonadio Jan 30 '14 at 20:14
  • Yes I do have a cron running. The site is slow in general. So many legacy code that is not worth re factoring. – awm Jan 30 '14 at 22:21
  • I agree with @MPD here, I don't think this is memory related. MySQL is also just one of the possible reasons. There's only one way to find out and that's to profile it. The easiest way to do that is using the xhprof extension and devel, that also works with drush (use -d to see the link to the report). My guess is that there are a number of issues, A typical problem if the site is slow for all requests is missing modules, see drupal.stackexchange.com/questions/724/why-is-drupal-7-so-slow. Views also has some major performance issues with caches, see drupal.org/node/1944674. – Berdir Feb 5 '14 at 8:44
  • Thank you, I figured that I have to do profiling, but in the case of drupal code base, it's always hard to pinpoint the bottleneck. Although I said the site is slow, it is not too slow and drush cc all, is unproportionally slower. – awm Feb 6 '14 at 17:09
6

The clearing cache it-self doesn't take a long time, since it's just truncating cache tables. What takes time the most is rebuilding the cache registry after that. Usually it's theme registry which scanning for all new hooks and all your Drupal folders for new template files, system which scanning files for the new modules and class files, and similar.

You can always clear specific cache by specifying drush cc theme-registry or any other.

It's highly recommended that you use PHP caching mechanism (e.g. OPCache, XCache, etc.) to speed up the processing. And memory-base cache to replace heavy usage on SQL tables (e.g. memcached or redis), so clearing cache would take no time, just by flushing the cache (e.g. echo flush_all > /dev/tcp/127.0.0.1/11211 in Bash).

Alternatively you can always clear cache manually, e.g. by:

echo "SHOW TABLES LIKE 'cache%'" | $(drush sql-connect) | tail -n +2 | xargs -L1 -I% echo "DELETE FROM %;" | $(drush sql-connect) -v

Debugging

To check what's taking the most time specifically, you need to debug/profile it (e.g. XDebug, XHProf, phpdbg, dtrace).

On OS X/Unix, this can be achieved by dtrace (after running drush):

sudo dtrace -qn 'php*:::function-entry { printf("%Y: PHP function-entry:\t%s%s%s() in %s:%d\n", walltimestamp, copyinstr(arg3), copyinstr(arg4), copyinstr(arg0), basename(copyinstr(arg1)), (int)arg2); }'

On Linux, use strace, e.g.

strace -e trace=sendto,recvfrom -s1000 -p $(pgrep php)

To look for some specific things, try adding e.g.: strace ... 2>&1 | grep -C5 UPDATE.

5

If you have a really large database, and your system is running fine when you are not doing a cache clear, it is likely that you do not have enough memory to fully support your setup.

If your site is running on a Linux box, run 'top' (from your shell) and press shift-M to sort the process list by memory used. Then, run your cache-clear operation from another terminal. You should see mysql and apache rise to the top of the list. You'll be able to see what percentage of total memory each of these processes are using, and how much free RAM is used. If you have a large amount of virtual space, but all of your physical RAM is exhausted, this operation could be causing VM memory to thrash, which can drive your execution time down to a small fraction of what it normally is.

Once, I was running a mid-traffic Drupal site on a box w/out -quite- enough memory to support the setup. When I ran a cache-clear on an unrelated low-traffic site, the cache rebuild pushed the system past its limit, and everything all but locked up. So, the total system behavior is important here; this is why simple tools like 'top' are a handy place to start.

  • Thanks, I think the underlying cause is that the site has been around for a while, the db is a bit large and the code terribly needs re factoring. For example, a node_delete operation takes about 3 seconds. I think giving too much memory may be redundant, do you agree? – awm Jan 30 '14 at 22:23
  • I have it on my machine and it is also slow. I doubled the memory, up to 1GB and timed it ` time drush cc all`. Not much better only got 4s faster. – awm Jan 30 '14 at 22:38
  • 1
    Yes, if the whole site is slow all the time even with a lot of memory, then cc is not the problem; you'll have to do more general profiling. – greg_1_anderson Jan 30 '14 at 23:25
  • Also, if the site is really old and needs a refresh, consider rebuilding from scratch with fresh modules, and use drupal-to-drupal migration (drupal.org/project/migrate_d2d) to move your content over. – greg_1_anderson Jan 30 '14 at 23:27
2

I am going to disagree (somewhat) with @greg_1_anderson here.

If the system isn't crashing totally during a cc all, then I don't think you have a general memory issue. When a LAMP server runs out of memory, it will hit swap. An active server hitting swap will cause an avalance of badness. httpd processes will start to stack up due to system slowdown (swap makes a system run very slow), which will cause more swap to be used, etc. On sites where I have seen this happen, I would see the process load hit 100, and a ton of active httpd processes.

If your system eventually comes back, then I think you are poorly tuned. drush cc all will result in a lot of database accesses, so I think it is showing the problem more. My suggestion would be to run mysqltuner on the site. If you have a multi GB database, my guess is that your innodb_buffer_pool_size isn't even remotely sized properly, and your MySQL instance is thrashing. I would also investigate an alternate cache backend to try to keep the database footprint smaller.

  • Thanks, drupal is used only for editoria use that has a servie layer which has varnish infornt of it. Users do now get to drupal. I just want to investigate whats happening because I think there is bottleneck. I think my best bet is to turn on the mysql slow log and monitor the db. And then do some profiling with xdebug – awm Feb 1 '14 at 21:36
  • SHOW FULL PROCESSLIST will tell you what's happening without need to use slow query log (and so without server restart). See other answer for mytop as well. – Chris Burgess Feb 4 '14 at 21:30
1

It could be your web hosting environment. Are you referring to a local setup, or something hosting on shared hosting or a VPS/server?

  • hosting environment - if you are on shared web hosting, the amount of memory Drupal/drush can use will be limited see: https://drupal.org/node/207036
  • max execution time - needs to be increased
  • drush normally isn't limited by max_execution_time. You can do drush php-eval "print ini_get('max_execution_time');" to double check, though. – mpdonadio Jan 31 '14 at 17:16
1

This is not a complete solution, just one more tool to help identify the source of your delays.

As well as using top to monitor processes, you might find the output of mytop informative. (Other answers above presume MySQL but if you're using another DB backend, you'll need to swap mytop for an equivalent tool.)

mytop simply executes MySQL SHOW FULL PROCESSLIST in a loop, and shows you which queries are being executed (so which take a lot of time). If the cache clear is taking a long time cleaning up this or that table, you'll see exactly what's holding things up here. If you don't have access to install mytop, just do a crude version in your shell -

while true; do mysql -e 'SHOW FULL PROCESSLIST' && sleep 5 && clear ; done

If the delay is not stemming from MySQL queries, then this tool may at least confirm that for you.

1

I found a blog post entitled Speed Up Cache Clearing on Drupal 7 to be very helpful in narrowing down precisely what portion of the cache clear process was slowing things down. The issues it pinpoints in features module and the entity api module were affecting my site, but the process detailed in the post also helped me track down issues we were encountering in Drupal core and breakpoints module.

It takes some time to work through the process, but it has helped me to reduce cache clears down from multiple minutes to under a minute.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.