I have created a Drupal 7 module that utilizes hook_cron() and hook_cron_queue_info() to process some data, as well as use the module Ultimate Cron to handle scheduling when each runs.
hook_cron() is run every 15 minutes and it's job is to determine how much data needs to be processed and split it up into batches, creating a queue for each batch. As an example, say 1000 records, split into batches of 25, so you get 40 queues of 25 records each.
The queue is then processed every minute, so every minute it will process one queue item of 25 records.
The problem I am running into is that occasionally during the worker callback for a queue, when processing one or more of the 25 records, a record may result in an exception being thrown.
The exception is possible and expected, so I am handling the exception in a simple try/catch, so the queue is able to continue on and processes all 25 records. However it seems that if any of the 25 had an exception, the queue item does not get removed at the end like it does when there no problems. Ultimate Cron shows the queue as having started, but never finished and is still running (though really it is not, it got through all 25), and this prevents any further queue items from running.
This is not the behavior I would like, I want the queue item to be removed or "finished" regardless if an exception happened, because I have handled the exception(s).
I am unsure of what is deciding the queue should not be removed, is it drupal's queue api? Ultimate cron? Is there a way I can have it remove the queue manually at the end of the batch if there happened to be an exception?
Basic code:
/**
* Implements hook_cron().
*/
function MODULENAME_cron() {
MODULENAME_queue();
}
/**
* Implements hook_cron_queue_info().
*/
function MODULENAME_cron_queue_info() {
$queues['MODULENAME_queue'] = array(
'worker callback' => 'MODULENAME_process_batch',
);
return $queues;
}
/**
* Queue up batches.
*/
function MODULENAME_queue() {
// Get the queue.
$queue = DrupalQueue::get("MODULENAME_queue");
// Only run if there are no items in the queue.
if (!$queue->numberOfItems()) {
// Get the full list of records.
$records = MODULENAME_get_records();
// Split the list of records into batches to process.
$pager = 25;
$batches = array_chunk($records, $pager);
// Queue the batches.
foreach ($batches as $batch) {
$queue->createItem($batch);
}
}
}
/**
* Process a batch.
*/
function MODULENAME_process_batch($data, &$context = NULL) {
// Process a batch of 25 records.
foreach ($data as $record) {
MODULENAME_process_record($record);
}
}
/**
* Process a single record.
*/
function MODULENAME_process_record($record) {
// Processing code here, this is where an exception can happen.
}
