I'm trying to make drush run my batch jobs. There's some amount of documentation at Processing batch jobs with drush, however, I can make no sense of it.

I would like to see real code, and the CLI requirements/instructions for running it, that when called by Drush, prints something trivial, such as "Hello world", once per operation.


I've written drush CLI commands. I've also written some Batch API processes. On each individual topic you've asked about there is alot of documentation.

I'm not sure why you want to use the Batch API specifically via Drush. The Batch API in part is used to avoid HTTP request timeout in long PHP processes. If you're using Drush from the command line -- you have no max_execution_time to worry about from php.ini essentially ...

EDIT: Have you looked into the Migrate source to see how they run Batch stuff from drush cli?

  • 1
    For one use case, see the recent blog post on Drupal Planet about getting ~8x speedup on batch jobs by running them through Drush using Drush backend_invoke_concurrent: dominiquedecooman.com/blog/… See also: drupal.org/node/1797900 Oct 3 '12 at 19:49
  • Re: "I'm not sure why you want to use the Batch API specifically via Drush" – Another use case: Let's say you are developing a batch script for updating all nodes on a site, and you want to find the optimal number of nodes to process per page request. Then you could write drush scripts with and without batch API, and time or profile those. Sep 17 '16 at 8:50

Look at the Drush core updatedb command - this uses the batch system. You can also look at the Drupal 7 content_migrate Drush command that is part of the CCK project.

The main reason you would want to use the batch system from Drush is that it runs each batch in a separate command line process. This gives some ability to manage fatal errors (very handy when developing upgrade/migrate type systems, so you can still complete as much as possible and then go back and review errors!), as well as to constrain memory - even with an unlimited timout and unlimited memory in php.ini, Drupal static caches and other things will eventually use up all the memory on a server (I have used 8GB or more in the past) on runs that are long/complex enough.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.