0

For a site I'm running, I'm going to have a few different queues for message sending. For example: high priority email, low priority email, high priority twitter, low priority twitter... etc.

I have set up queues (like "UrgentEmail", "UrgentTwitter") etc. using Drupal's Queue API, but now I need to work on the queue processing (consuming).

I can simply use hook_cron in a module to consume one queue at a time, or all of them, as much as possible, but I'd like to be able to maybe have one queue get hit every couple minutes, while the lower-priority queues would only get it once every half an hour or so...

Is this possible without writing up some of my own cron handlers? I know elysia cron for D6 was good at setting options for module-specific cron, but is there a way I could branch out cron in a single module and run different tasks at different intervals? Or is there another Drupal Way™ to do this?

3 Answers 3

1

Drupal does not offer anything like this out of the box.

drupal.org uses Jenkins, which then starts drush scripts. Jenkins is very flexible to configure and has good built-in reporting while drush scripts make it easy to run code inside the context of drupal, without having to worry about authentication (as opposed to accessing it over HTTP, for example).

So, in your case, you could create a number of drush scripts (e.g. a script pro type like twitter/mail + an argument for the urgency) and then configure jenkins to run them at various time intervals.

3
  • Sounds like a reasonable way of doing things. Didn't think at all about using drush. I might look into running things by drush alone (if that's possible), as I don't need the benefits of the Jenkins dashboard as much as I simply need a way to set jobs to run at different intervals (set once and forget). Jun 3, 2011 at 0:30
  • 1
    You can of course also start drush scripts directly with cron, that's how I run cron for my drupal sites too.
    – Berdir
    Jun 3, 2011 at 0:35
  • Thanks again for this suggestion. I'm writing a drush command to do exactly what I need, and it looks like this is going to be, by far, the best and most customizable route. Jun 3, 2011 at 15:02
2

It's possible to achieve this using Ultimate Cron! module.

This module creates a separate entry for each Queue item. Maybe it will help somebody.

1

It's not possible to run cron queues at different times, but it's possible to assign to each queue the time spent from Drupal to run it.

function mymodule_cron_queue_info() {
  $queues['mymodule_low_priority'] = array(
    'worker callback' => 'mymodule_low_priority_worker', 
    'time' => 5,
  );

  $queues['mymodule_high_priority'] = array(
    'worker callback' => 'mymodule_high_priority_worker', 
    'time' => 45,
  );

  return $queues;
}

Both the queues will be executed during cron tasks, but assigning less time to the one you consider a low priority queue means that the queue will take more cron tasks to be completed, if all the queues take the same time to be completed. If the queue you consider low priority contains only two items, then it will be completed before other queues that contain more items.

7
  • Interesting... yet another hook I had never seen before. That's two new things learned on this site in one day. I'm sorely tempted to mark this as the answer, as it's all built-in drupal core, but I'll just give it my plus 1. Very good to know: api.drupal.org/api/drupal/modules--system--system.api.php/… Jun 3, 2011 at 2:35
  • It looks like Job Scheduler (drupal.org/project/job_scheduler) might also be an option here. Jun 3, 2011 at 2:46
  • @geerlingguy It seems so. The module is based on the Drupal Queue API.
    – apaderno
    Jun 3, 2011 at 3:04
  • 1
    Note that while this looks useful, it is not what you asked for :) time specifies how long each queue is processed. Every registered queue is run on every cron run, without any priorization or global timeout. It might be possible to mess with that by only returning some of the queues in the info hook, but that doesn't sound like a good idea to me... Job Scheduler will probably work better but is still tied to the cron schedule, unless using something like elysia cron.
    – Berdir
    Jun 3, 2011 at 7:18
  • @Berdir That is what I wrote in the first sentence of my answer. Giving to a queue less time slots is the only way of categorize a Drupal queue as low priority. If you have a task to do, and you dedicate less time to that task than for other tasks, does that mean you consider that task a high priority one?
    – apaderno
    Jun 3, 2011 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.