How to check if an item exists in a Queue?

Question

While inserting items in a Queue how can I check if the item I am trying to insert is already present in the Queue?

Answer 1

The queue API doesn't have a method for this. It is very limited on purpose, as some queue systems might not be able to provide information like this.

The update module has its own wrapper to do this, by additionally storing all the task names in key value as well. See \Drupal\update\UpdateProcessor::createFetchTask:

public function createFetchTask($project) {
  if (empty($this->fetchTasks)) {
    $this->fetchTasks = $this->fetchTaskStore->getAll();
  }
  if (empty($this->fetchTasks[$project['name']])) {
    $this->fetchQueue->createItem($project);
    $this->fetchTaskStore->set($project['name'], $project);
    $this->fetchTasks[$project['name']] = REQUEST_TIME;
  }
}

Answer 2

Add this function:

/**
 * Adding item to queue.
 *
 * @param string $queue_id
 *   The name of the queue to work with.
 * @param array $item_data
 *   Arbitrary data to be associated with the new task in the queue.
 * @param bool $find_identical
 *   Checking exist item in the queue or not.
 *   If exists, new item wouldn't be added in queue.
 */
function MY_MODULE_add_to_queue($queue_id, array $item_data, $find_identical = FALSE) {
  $queue_item_exists = FALSE;

  if ($find_identical) {
    $query = \Drupal::database()->select('queue', 'q');
    $query->condition('name', $queue_id);
    $query->condition('data', serialize($item_data));
    $query->condition('expire', 0);
    $query->fields('q', ['item_id']);

    $queue_item_exists = !empty($query->execute()->fetchAll());
  }

  if (!$queue_item_exists) {
    \Drupal::queue($queue_id)->createItem($item_data);
  }
}

And in place where you adding new items in queue use this:

MY_MODULE_add_to_queue('YOUR_QUEUE_WORKER', $item_to_add, TRUE);

Answer 3

It depends on how your implementation. How do you define uniqueness? By a duplicate queue item object (e.g. all properties are equal), by a specific property on your queue item object, or something else?

The Queue interfaces QueueInterface & ReliableQueueInterface don't define any method for uniqueness. It's up to you to define what unique means and to enforce it when using the createItem() method.

To have a concrete example, take a look at the Queue Unique project (it's D7 but the premise is the same this has releases for D7 and D8/D9 now). It's creating it's own implementation of DrupalQueueInterface (QueueInterface in D8) and adding a table where it MD5ing a serialized queue object to determine if an item is unique (if the MD5 hash of the serialized object is the same, it's a duplicate which is rejected).

Answer 4

Here you can find a nice D7 solution: https://www.webomelette.com/overriding-queues-drupal-7-or-how-not-create-duplicate-queue-items