20

I've got around 70k nodes of specified type on my site. I need to run an update on them. Some operations and setting one field to desired value. node_save is really slow and it causes crashes (too long callstack mayby). Is there faster way to write info on this one particular field?

There was field_attach_update mentioned in one post, but it isn't much faster.

EDIT: There is quite complex view built on this node type, but is not working on this field which I want to update.

0

7 Answers 7

31

I'd definitely go for field_attach_update.

The idea is simple. Just load the node and save it using field_attach_update.

Ex:

$node = node_load($nid);
$node->field_name[LANGUAGE_NONE][0]['value'] = 'New value';
field_attach_presave('node', $node);
field_attach_update('node', $node);
  // Clear the static loading cache.
entity_get_controller('node')->resetCache(array($node->nid));

This will not changed any timestamp or any other hook that node_save usually invokes. Loading the node will also invoke some hooks so probably it's not that efficient.

If you have the nid and if the node structure is dead simple, you can do it like this as well:

 $node = new stdClass();
 $node->nid = $nid; // Enter the nid taken. Make sure it exists. 
 $node->type = 'article';
 $node->field_name[LANGUAGE_NONE][0]['value'] = 'New value';
 field_attach_presave('node', $node);
 field_attach_update('node', $node);
  // Clear the static loading cache.
 entity_get_controller('node')->resetCache(array($node->nid));

Anyways, if you are trying to update anything other than than fields, this will no work (comment status, published status, etc). Also If you are using node_save, cache for the particular node will be cleared automatically for different methods we need to clear it with 'entity_get_controller'.

Update: It appears that you should also call field_attach_presave() to let other modules process the field input properly. File module, for example, uses it to set the file status to permanent using this hook. I have updated my 2 examples above.

2
  • I do have nid but structure is not so simple for node, but field I want to update is dead simple. What I could expect from setting only one field for existing node (identified by nid but not loaded) and then calling field_attach_update?
    – Eloar
    Mar 8, 2013 at 7:40
  • 4
    As it turnd out using entity query was slowing things down. So switching from node_save to field_attach_update and from EntityFieldQuery to db_query_range was quite rewarding. From 3h update to 40 minutes.
    – Eloar
    Mar 10, 2013 at 23:56
3

Having tried all the approaches mentioned in the other answers I got very slow update times (about 7 days for 700.000 nodes of a node type with 20+ fields) until I found this article: http://www.drupalonwindows.com/en/blog/only-update-changed-fields-or-properties-entity-drupal .

After implementing something like the below code in a hook_update I reduced the update time to 2 hours, which I think is manageable.

if (!isset($sandbox['storage']['nids'])) {
    $sandbox['storage']['nids'] = [];
    $query = 'SELECT {nid} FROM node WHERE type = \'article\';';
    $result = db_query($query)->fetchCol();
    if ($result) {
      $sandbox['storage']['nids'] = $result;
      $sandbox['storage']['total'] = count($sandbox['storage']['nids']);
      $sandbox['storage']['last_run_time'] = time();
      $sandbox['progress'] = 0;
    }
  }

  $amount = 300;
  $nids = array_slice($sandbox['storage']['nids'], 0, $amount);

  if (!empty($nids)) {
    $nodes = node_load_multiple($nids, [], TRUE);
    foreach ($nodes as $node) {
      // Lets manipualte the entity.
      $article_wrapper = UtilsEntity::entity_metadata_wrapper('node', $node);
      // Eventual logic here.

        // Field to update
        $article_wrapper->my_field = 'my_value';
        $article_wrapper->save();

    $sandbox['progress']++;
    }
    $sandbox['message'] = 'Runs left: ' . (($sandbox['storage']['total'] - $sandbox['progress'])/$amount) . ' Progress: ' . (($sandbox['progress'] * $amount)/$sandbox['storage']['total']) . '%';
    $sandbox['storage']['last_run_time'] = time();
    unset($nids);
  }
  $sandbox['storage']['nids'] = array_slice($sandbox['storage']['nids'], 100, count($sandbox['storage']['nids']));
  if (!empty($sandbox['storage']['total'])) {
    $sandbox['#finished'] = ($sandbox['storage']['total'] - count($sandbox['storage']['nids'])) / $sandbox['storage']['total'];
  }
  return $sandbox['message'];
2

If you don't want to save field data without causing the standard events and actions to happen then you can use drupal_write_record.

Here is an example to insert Hello World into the body field for a node of type article with an id of 1.

$values = array(
  'entity_type' => 'node',
  'bundle' => 'article',
  'entity_id' => 1,
  'revision_id' => 1,
  'language' => 'und',
  'delta' => 0,
  'body_value' => 'HELLO WORLD',
  'body_summary' => '',
  'body_format' => 'filtered_html',
);
drupal_write_record('field_data_body', $values);
drupal_write_record('field_revision_body', $values);

If your site is multlingual then you will want to use 'en' or the language of your content instead of 'und'.

If you are doing revisioning, you will have to be careful to insert the right revision id, otherwise you can simply insert the same value as the entity_id.

Note how this data is inserted into two tables field_data_* and field_revision_*. You should insert into both to make sure the site works as desired.

After running this you will then need to clear the caches for the fields to show up depending on how your caching is setup.

2

For a simple update like this where lots of nodes need to be updated, I always use a MySQL update statement. Yes, caching needs to be taken into consideration but you can just flush the cache after you're done and it is all good. You, of course, need to be familiar with the data structure but it is relatively simple in Drupal 6. (although horrendous in Drupal 7)

1
  • Project was created for Drupal 7. After all some parts of my module was made to manipulate directly on Drupal DB structure for reading values and searching as sql queries were much faster than EntityFieldQueries.
    – Eloar
    Aug 17, 2014 at 20:34
2

I suggest field_attach_update too, and not a direct SQL query, because sql doesn't update the node cache object, and in your next node_load you wont load the updated field value, you will load the old value

field_attach_update is much better than direct SQL query.

3
  • Ayesh K suggested creating node as stdClass object without loading. Do you know what might happen if I try to update node this way without setting all fields? Will those be overwritten with nulls or defaults? Mayby those would be ignored by updating process?
    – Eloar
    Mar 10, 2013 at 23:54
  • 1
    It's possible to save a node directly by Ayeshs method (new stdClass etc...) only the UI makes validates on required fields
    – pico34
    Mar 11, 2013 at 8:25
  • I'll try to update node with this method without setting all fields and check what will happen. It might be fastest method for updating nodes in module update procedure (batch).
    – Eloar
    Mar 11, 2013 at 12:46
1

I even had the same requirement of updating a field for all the nodes of a particular content type. I used node_load_multiple and field_attach_update.

$nodes = node_load_multiple(array(), array('type' => 'content_type_name'));
foreach ($nodes as $node) {
  $node->field_name['und'][0]['value'] = 'field value';
  field_attach_update('node', $node);
}

I ran it through drush and it was pretty quick.

0

Have you considered doing these updates directly into the database using mySQL? It's probably the simplest & fastest way to achieve what you want.

Here is a simple example. You can execute such a command from the 'SQL' tab in phpMyAdmin. Imagine you have a content type called Member Profile. In it you have a field named 'Type of member' (e.g. company, individual, organisation). Let's say you want to update all occurrences for 'COMPANY' to 'company'. The following command will do just that.

UPDATE content_type_member_profile SET field_type_of_member_value = 'company' WHERE field_type_of_member_value = 'COMPANY';

Also, checkout Get Started with MySQL

3
  • It is one of options. I leave it as last one to check and implement. I don't want to mess up drupal structure and flaws to much. So first I look for any solutions directly through Drupal API. If you could provide some examples, it would be much appreciated.
    – Eloar
    Mar 8, 2013 at 7:37
  • Goog point. I added a simple example to my initial answer.
    – Bisonbleu
    Mar 8, 2013 at 13:35
  • eh, I do know SQL well enough to update any records in any tables. It is not the problem. The problem is, I'm not familiar enough with the way drupal stores data (especially meta-data). There is always lot of caching, preserving and so on in such systems, so updating it on the lowest level might (not always) mess up some flaws. So if it came to this, I will try to do it on lowest level, and I will be ready for some real mess.
    – Eloar
    Mar 10, 2013 at 23:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.