0

For a site I'm running, I'm going to have a few different queues for message sending. For example: high priority email, low priority email, high priority twitter, low priority twitter... etc.

I have set up queues (like "UrgentEmail", "UrgentTwitter") etc. using Drupal's Queue API, but now I need to work on the queue processing (consuming).

I can simply use hook_cron in a module to consume one queue at a time, or all of them, as much as possible, but I'd like to be able to maybe have one queue get hit every couple minutes, while the lower-priority queues would only get it once every half an hour or so...

Is this possible without writing up some of my own cron handlers? I know elysia cron for D6 was good at setting options for module-specific cron, but is there a way I could branch out cron in a single module and run different tasks at different intervals? Or is there another Drupal Way™ to do this?

1

Drupal does not offer anything like this out of the box.

drupal.org uses Jenkins, which then starts drush scripts. Jenkins is very flexible to configure and has good built-in reporting while drush scripts make it easy to run code inside the context of drupal, without having to worry about authentication (as opposed to accessing it over HTTP, for example).

So, in your case, you could create a number of drush scripts (e.g. a script pro type like twitter/mail + an argument for the urgency) and then configure jenkins to run them at various time intervals.

  • Sounds like a reasonable way of doing things. Didn't think at all about using drush. I might look into running things by drush alone (if that's possible), as I don't need the benefits of the Jenkins dashboard as much as I simply need a way to set jobs to run at different intervals (set once and forget). – geerlingguy Jun 3 '11 at 0:30
  • 1
    You can of course also start drush scripts directly with cron, that's how I run cron for my drupal sites too. – Berdir Jun 3 '11 at 0:35
  • Thanks again for this suggestion. I'm writing a drush command to do exactly what I need, and it looks like this is going to be, by far, the best and most customizable route. – geerlingguy Jun 3 '11 at 15:02
2

It's possible to achieve this using Ultimate Cron! module.

This module creates a separate entry for each Queue item. Maybe it will help somebody.

1

It's not possible to run cron queues at different times, but it's possible to assign to each queue the time spent from Drupal to run it.

function mymodule_cron_queue_info() {
  $queues['mymodule_low_priority'] = array(
    'worker callback' => 'mymodule_low_priority_worker', 
    'time' => 5,
  );

  $queues['mymodule_high_priority'] = array(
    'worker callback' => 'mymodule_high_priority_worker', 
    'time' => 45,
  );

  return $queues;
}

Both the queues will be executed during cron tasks, but assigning less time to the one you consider a low priority queue means that the queue will take more cron tasks to be completed, if all the queues take the same time to be completed. If the queue you consider low priority contains only two items, then it will be completed before other queues that contain more items.

  • Interesting... yet another hook I had never seen before. That's two new things learned on this site in one day. I'm sorely tempted to mark this as the answer, as it's all built-in drupal core, but I'll just give it my plus 1. Very good to know: api.drupal.org/api/drupal/modules--system--system.api.php/… – geerlingguy Jun 3 '11 at 2:35
  • It looks like Job Scheduler (drupal.org/project/job_scheduler) might also be an option here. – geerlingguy Jun 3 '11 at 2:46
  • @geerlingguy It seems so. The module is based on the Drupal Queue API. – kiamlaluno Jun 3 '11 at 3:04
  • 1
    Note that while this looks useful, it is not what you asked for :) time specifies how long each queue is processed. Every registered queue is run on every cron run, without any priorization or global timeout. It might be possible to mess with that by only returning some of the queues in the info hook, but that doesn't sound like a good idea to me... Job Scheduler will probably work better but is still tied to the cron schedule, unless using something like elysia cron. – Berdir Jun 3 '11 at 7:18
  • @Berdir That is what I wrote in the first sentence of my answer. Giving to a queue less time slots is the only way of categorize a Drupal queue as low priority. If you have a task to do, and you dedicate less time to that task than for other tasks, does that mean you consider that task a high priority one? – kiamlaluno Jun 3 '11 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.