1

I have a custom content type called a Scan where the only required field is ISBN. Nodes of this type are created when a user POSTs to my REST API endpoint with an ISBN number. That's all working fine, nice and quick (which it needs to be).

Now I want some additional process that -- in it's own time without blocking the API request -- goes over these newly created scans, querying an API (Google Books/ISBNdb/OpenLibrary) to fill out the other fields. I can confidently write the plain PHP to do this but I'm trying to learn how I would do this in the Drupal spirit?

  • I have learned custom modules so I've got an idea my code is going to sit in the controller (Yes?).
  • I have looked at Rules & Actions but these aren't working in Drupal 8 yet (Right?).
  • Cron seems to only run every few hours which would be too slow as I want my code to run within minutes of node creation.
  • I found hook_ENTITY_TYPE_create but not sure exactly how to create one? In my custom module?

Please help :-)

Disclaimer: I am new to Drupal and am self-teaching Drupal version 8. I am a senior PHP developer with a good understanding of good-old OO PHP but have just never done Drupal. Be gentle with me.

1

One easy way would be to add a hook_cron to your custom module:

https://api.drupal.org/api/drupal/core%21core.api.php/function/hook_cron/8.2.x

These get run when drupal runs it's periodic cron hook.

Then you could query for nodes that have the ISBN field but lack some other field you would fill-in with a second API call. So you could get a list of nodes that need updating with a query and then run your code to update them.

If you want it to run more frequently you could set a crontab on your server to hit a page callback that triggers a script. So set up a custom route using mymodule.routing.yml and set it up to require a secret key (to prevent DOS attacks), so for example ?accesskey=12345, and if that page is hit with the access key run your script to update the nodes. This way you could run it every minute.

Another alternative to creating a page callback is to create a custom drush command called from a crontab on your server. Creating custom drush commands is very easy.

Another alternative to using a query to identify items that need updating is the queue api. This would allow you to set up a list of items to process, although it doesn't process the queue, you'd still need some sort of cron job to do that. https://api.drupal.org/api/drupal/core!core.api.php/group/queue/8.2.x Although a query to find items that need processing from a cron process might be easier in this case.

I would recommend against calling APIs using hook_ENTITY_TYPE_create or hook_ENTITY_TYPE_update. If the API call fails, you can't edit the content. Better to offload it to a process not connected to a page load, such as cron, or a process called from a custom cron job.

For examples of cron hooks look at function hook_cron() in /core/core.api.php

https://api.drupal.org/api/drupal/core!core.api.php/function/hook_cron/8.2.x this has examples, of using a Drupal queue as well as an entity query.

Here's an example of a type of query where you'd check isbn to see if nodes with isbn filled out have empty data in another field, say, field_api_title:

$db = Drupal::database();


$query = $db->select('node__field_isbn_longer', 'isbn')
   ->fields('isbn', array('entity_id'));

      $query->leftJoin('node__field_api_title', 't', 't.entity_id = isbn.entity_id');
      $query->condition('tags.field_api_title_target_id', NULL, 'IS NULL');


$nids = $query->execute()->fetchCol();

A left join allows you to join to tables that are missing data. It will return rows where there is an ISBN, but the api title hasn't been retrieved. Then you can run your code to update those nodes.

  • Thank you for such a thorough answer! I have just whipped up a method called hook_cron that just sends an email for now, created a route to manually trigger it and it worked perfectly. Now, would you also be able to point me in the right direction for learning the proper way to get a bunch of nodes/entitys and modify them. I remember reading you should never query the DB directly in D8 but use an Entity API instead, right? – Martin Joiner Oct 7 '16 at 15:36
  • hook_cron shouldn't be the method name. HOOK_ should be replaced with your custom module name, such as mymodule_cron. You can use direct queries in Drupal 8. Where did you read that's not allowed? I'm curious. – oknate Oct 7 '16 at 15:41
  • It was in the comments section below an article. I've just had a browse through my history to see if anything jogs my memory but there is so many contenders. I've basically been reading/watching Drupal 8 tutorial all day, every day for over a week. – Martin Joiner Oct 7 '16 at 15:52
  • Oh, I'm looking through the code base and I see it does say hook_cron in the hook. For example: "statistics_hook_cron" – oknate Oct 7 '16 at 16:00
  • Updated the answer, check out api.drupal.org/api/drupal/core!core.api.php/function/hook_cron/…, it has examples of queries and using queue. – oknate Oct 7 '16 at 16:04
4

As an alternative to the cron job, this sounds like the queue system might be better.

The main difference is that in your hook_cron(), you have to execute queries to find data to process. With a queue, you listen to new entities being saved (hook_entity_insert()) and put them in your queue.

Then Drupal will manage the list of tasks (queue items) and will automatically invoke your callback for each of them. It will also handle time management and process the configured amount of queue items until the time you configure on your queue is up.

There are plenty of examples and blog posts to be found, this one seems pretty good: https://www.sitepoint.com/drupal-8-queue-api-powerful-manual-and-cron-queueing/. It seems to be very close to your use case, with a hook_entity_insert() than then does stuff with the new entities. It's pretty complete and does some more advanced things like a form where you can check the pending items and fire them off manually. You can of course skip that if you don't need it.

  • I would add that the advantage of Queue API is that you don't have to run it through cron, but can leverage an external processor if necessary. – mradcliffe Oct 7 '16 at 18:34
  • right, if you use an alternative queue backend/processing system. By default, it's still cron. But especially when combined with ultimate cron, you can fire it as often as you want. And you can start simple with the standard queue and processing implementations and switch to more advanced and scalable solutions if you have a lot to process. – Berdir Oct 7 '16 at 18:38
  • Thank you for an alternative great answer. I will experiment with both techniques for the sake of expanding my learning of Drupal 8. – Martin Joiner Oct 8 '16 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.