9

I am using Drupal multi-sites (single codebase, multiple sites/*). Along with this, I have started using Drush aliases to manage them:

$ cat sites/all/drush/aliases.drushrc.php
<?php
$aliases['localdev'] = array(
  'site-list' => array(
    'site1', 
    'site2',
    'site3',
  ),
);
?>

This allows me to easily perform actions on all sites:

$ drush @localdev cc all

>> I've also just discovered that I can just use @sites, and forgo the drushrc file.

Doing this, will execute "cc all" on each of my sites in series (one at a time).

I'd like to take this to the next level and try to run these commands on all sites simulantiously. I've been doing some reading, and am under the impression that Drush does indeed support this. The drush_invoke_process() function takes $backend_options, which can contain (from function documentation):

 *      'invoke-multiple'
 *        If $site_alias_record represents a single site, then 'invoke-multiple'
 *        will cause the _same_ command with the _same_ arguments and options
 *        to be invoked concurrently (e.g. for running concurrent batch processes).
 *      'concurrency'
 *        Limits the number of concurrent processes that will run at the same time.
 *        Defaults to '4'.

What I can't figure out, however, is how do I actually use this from the Drush command line. Is there an option I need to pass to Drush, or do I need to set something in a settings file?

Any info will be much appreciated - my curiosity is piqued!

UPDATE

Based on the answers below, I was able to create a simple test that demonstrates Drush's behaviour, and draw some conclusions:

Drush's default behaviour when executing operations on multiple sites is to use concurrent processes:

$ drush @localdev ev "drupal_set_message(time()); sleep(5);"

Continue?  (y/n): y
site1             >> 1360512943      [status]
site2             >> 1360512943      [status]
site3             >> 1360512943      [status]

This is true even when not using aliases, and is also true when using Drush's built-in @sites alias. These two commands yield identical behaviour as above:

$ drush site1,site2,site3 ev "drupal_set_message(time()); sleep(5);"
$ drush @sites ev "drupal_set_message(time()); sleep(5);"

To change the number of concurrent processes (default is 4), the '--concurrency=N' option can be passed in the drush command. For example, if I want serial execution, I can set the number of concurrent processes to 1:

$ drush @localdev ev "drupal_set_message(time()); sleep(5);" --concurrency=1

Continue?  (y/n): y
site1             >> 1360513387      [status]
site2             >> 1360513393      [status]
site3             >> 1360513399      [status]
  • That's a very good summary; thanks for writing it up. It would be great if that info was in the Drush documentation somewhere. I opened an issue to capture that: drupal.org/node/1914224 – greg_1_anderson Feb 11 '13 at 15:17
5

This worked for me:

drush @site1,@site2,@site3,@site4 cc all --concurrency=4

I'm not sure how concurrent it actually was; the last message about site1 came immediately after the first message for site2, and all other messages were printed sequentially. I did not measure to what extent each cc operation happened concurrently, or to what extent the system might have just been cpu or i/o bound, but it did seem to be nominally working.

  • I am working with something similar to this and realized a handy way of doing things using the @sites command. However, one glitch with it is that if the site directory is a symlink, the command does not recognize it. in my case the symlink is to a dir outside drupal root so ls- l gives :site_dir -> ../../sites/site/src.. perhaps it's bug I can fix if you can point me to the code responsible of building thelist – awm Jan 9 '17 at 17:37
1

For single instance (without site-list):

<?php
$aliases['localdev'] = array(
  'invoke-multiple' => TRUE,
);
?>

For aliases with site-list array it will run concurently even...

After below comments, let's review code for drush_invoke_process:
// - my comment, /* ... */ - shortening provided code.

<?php
function drush_invoke_process($site_alias_record, $command_name, $commandline_args = array(), $commandline_options = array(), $backend_options = TRUE) {
  if (is_array($site_alias_record) && array_key_exists('site-list', $site_alias_record)) {
    /*  $invocations[] - this array filled with command for each site in site-list. */
  }
  else {
    /* aliases not defined or site-list not found.  So $invocations filled by one item. */
  }
  return drush_backend_invoke_concurrent($invocations, $commandline_options, $backend_options);
}
?>

Next called:

<?php
function drush_backend_invoke_concurrent($invocations, $common_options = array(), $common_backend_options = array(), $default_command = NULL, $default_site = NULL, $context = NULL) {
  /* Here building command line happen for each site (invocation). */
  return _drush_backend_invoke($cmds, $common_backend_options, $context);
}
?>

Next will called:

<?php
function _drush_backend_invoke($cmds, $common_backend_options = array(), $context = NULL) {
  /* Some simulating code and fork code */
  if (array_key_exists('interactive', $common_backend_options) || array_key_exists('fork', $common_backend_options)) {
    /* Direct running (interactive or fork) */
  }
  else {
    // Concurrency set to 4 by default. So --concurency just override it by another value.
    $process_limit = drush_get_option_override($common_backend_options, 'concurrency', 4);

    // Next is main call, that run commands as concurent processes using proc_open and streaming:
    $procs = _drush_backend_proc_open($cmds, $process_limit, $context);

    /* Processing of result running of processes. */

  }
  return empty($ret) ? FALSE : $ret;
}
?>
  • Can you please clarify? Are you saying that when using a site-list, Drush will automatically execute the commands concurrently on all the sites? I'm confused because a Drush maintainer suggested that the default behaviour is serial execution drupal.org/node/628996#comment-2637008. – rcourtna Feb 10 '13 at 5:02
  • invoke-multiple is for running the same command on the same site with the same options and arguments multiple times. You want --concurrency=N for running the same command on multiple sites. That is the intention, anyway; I did not test with @sites or a 'site-list', but you're straying outside of intended behavior, should that happen to work. – greg_1_anderson Feb 10 '13 at 5:42
  • You are correct about --concurrency; if you run a command on multiple sites in debug mode without --concurrency and without --invoke-multiple, you can easily see that it is running all of the commands concurrently. But again, 'invoke-multiple' => TRUE does nothing, and setting it to 2 in a site alias would make all of your commands run twice. – greg_1_anderson Feb 10 '13 at 18:20
  • 2greg_1_anderson: invoke_multiple setting will work if you dont' set alias or site-list... – Nikit Feb 11 '13 at 1:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.