I'm creating a File Checker module that verifies files managed by Drupal actually exist at the uri where Drupal believes they are.
It places an item in a queue for each file entity. As there could be large amounts of files on a site, so there can be large amounts of queue items (10000, 100000 ...). Each item (file entity) can take a few seconds to process if the files are stored remotely e.g. on Amazon S3 or Dropbox. So the queue could take hours to process.
File checking can be scheduled to happen at regular intervals, i.e. by cron, so queue processing needs to happen by cron queueWorkers.
What is the best way to handle the duration of these cron queueWorkers? Their plugin annotation has a cron time option (default 15 seconds) that controls how long they run for when awoken by cron. I could either:
1) set their time option to something very long like 86400, so they can process queue items for a whole 24 hours if they need to; or
2) set their time short like 60 seconds, and use something like the Ultimate Cron module to specify that just this queueWorker is woken up every 60 seconds (and not other cron tasks).