How to write big updation code using Batch API?

Question

I am writing a script to update all website registered user's Username & Email address. I have more then 5000+ users in my website which causes PHP execution time out error.

So I tried to implement Batch API but I am unable to implement it. I passed whole user list to Batch API and perform operations on it. But Batch API also shows the same PHP time out error.

Can someone help me to implement Batch API?

Here is my code:

function anonymize_data_menu() {
  $items = array();
  $items['admin/config/content/anonymize-user-data'] = array(
    'title' => 'Anonymize User Data',
    'description' => 'Click this link to anonymize user data',
    'type' => MENU_LOCAL_ACTION,
    'page callback' => '_anonymize_users_data',
    'access arguments' => array('administer user anonymize'),
    'file' => 'anonymize_data.user.inc',
  ); 
  return $items; 
}

function _anonymize_users_data() {
  $users = entity_load('user');
  $chunks = array_chunk($users, 20);
  $count_user = count($chunks);

  foreach ($chunks as $chunk) {
     $operations[] =  array('user_anonymize_op', array($chunk));
      //user_anonymize_op($chunk);
     //krumo($chunk);
  }
  $batch = array(
    'operations' => $operations,
    'title' => t('Import batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('An error occurred'),
    'finished' => 'user_anonymize_op_finished'
  );
  batch_set($batch);
  batch_process('admin/people');
}

function user_anonymize_op($users, &$context) {
  foreach ($users as $user) {
    # code...
    //krumo($user);
   $context['message'] = t('processing user @user', array('@user' => $user->name));
    //
      if($user->uid != 0 && $user->uid != 1) {
      $user->name = $i . '-' . $user->name;
      $user->pass = user_hash_password('blah');
      user_save($user);
    }
  }
}

function user_anonymize_op_finished($success, $results, $operations) {
  if ($success) {
    $message = count($results) . ' processed.';
  }
  drupal_set_message(t($message));
}

I am very confused about that how to process chunk of users.

Can someone help me it will be great.

Answer 1

The timeout is caused because you're trying to load all 5K+ users before you start the batch. Instead get the User IDs first and load when processing. Check an example below. Also have a look at Batch API Overview documentation page, before you continue:

function anonymize_data_menu() {
  $items = array();
  $items['admin/config/content/anonymize-user-data'] = array(
    'title' => 'Anonymize User Data',
    'description' => 'Click this link to anonymize user data',
    'type' => MENU_LOCAL_ACTION,
    'page callback' => '_anonymize_users_data',
    'access arguments' => array('administer user anonymize'),
    'file' => 'anonymize_data.user.inc',
  ); 
  return $items; 
}

function _anonymize_users_data() {
  // Instead of loading all users at once,
  // first get there uid and then load chunk of user while processing.
  // $users = entity_load('user');
  $uids = db_select('users', 'u')
    ->condition('u.uid', 1, '>')
    ->fields('u', array('uid'))
    ->execute()->fetchAllKeyed(0, 0);

  $chunk_count = 20;
  // We've all rows here, split them in chunks.
  $chunks = array_chunk($uids, $chunk_count, TRUE);
  // Get all rows count.
  $num_operations = count($uids);

  $operations = array();
  foreach ($chunks as $chunk) {
    $operations[] =  array('user_anonymize_op', array(
      $chunk,
      array(
        'max' => $num_operations,
        'chunk' => $chunk_count,
      )
    ));
  }

  $batch = array(
    'operations' => $operations,
    'title' => t('Import batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('An error occurred'),
    'finished' => 'user_anonymize_op_finished'
  );

  batch_set($batch);
  batch_process('admin/people');
}

function user_anonymize_op($rows, $batch_data, &$context) {
  // Define variables on start of batch operation.
  if (!isset($context['results']['progress'])) {
    $context['results']['progress'] = 0;
    $context['results']['chunk'] = $batch_data['chunk'];
    $context['results']['max'] = $batch_data['max'];
    $context['results']['failed'] = array();
  }

  // Do your processing here. START HERE.
  $names = [];
  $users = entity_load('user', $rows);
  foreach ($users as $account) {
    if (!empty($account->uid) && ($account->uid > 1)) {
      // What is $i here?
      $account->name = $i . '-' . $account->name;
      $account->pass = user_hash_password('blah');
      user_save($account);
      $names[] = $account->name;
    }
  }
  // Do your processing here. END HERE.

  // Update completion count.
  $context['results']['progress'] = $context['results']['progress'] + count($rows);

  $context['message'] = t('Processed @name user data.', array('@name' => implode(', ', $names)));

  // Inform the batch engine that we are not finished,
  // and provide an estimation of the completion level we reached.
  if ($context['results']['progress'] != $context['results']['max']) {
    $context['finished'] = $context['results']['progress'] % $context['results']['max'];
  }
}

function user_anonymize_op_finished($success, $results, $operations) {
  if ($success) {
    $message = count($results) . ' processed.';
  }
  drupal_set_message(t($message));
}

Answer 2

First time it could seems to be a huge learning curve, but it isn't.

And once Batch API is learned, it is really powerfull and easy to implement.

Best way for starting, is looking at core example. The keyword you should look for is $sandbox. Which is the object that will be injected into each call of the batch. Often used on hook_update_n( $sandbox )

Here is a truncated example from system.install I commented out

I suggest to put your Batch into a hook_update and put your site in maintenance during the update process to prevent new user entry during the batch process, which could invalidate the batch somehow.

function system_update_8001(&$sandbox = NULL) {
      $database = \Drupal::database();
      $schema = $database->schema();

      // It checks if something has to be done
      if ($schema->tableExists('menu_tree')) {

            // If our sandbox object doesnt have the current property
            // It means this is the first call of the batch
            if (!isset($sandbox['current'])) {

                  [... truncated code ]

                  // So lets initiate the needed properties
                  $sandbox['current'] = 0;
                  $sandbox['max'] = $database->query('SELECT COUNT(mlid) FROM {menu_tree}')
                     ->fetchField();
           }

          // request the data from database
          // this is where $sandboc['current'] will be usefull 
           $menu_links = $database->queryRange('SELECT mlid, system_update_8001_title AS title, system_update_8001_description AS description FROM {menu_tree} ORDER BY mlid ASC', $sandbox['current'], $sandbox['current'] + 50)
             ->fetchAllAssoc('mlid');

          foreach ($menu_links as $menu_link) {

                // Process each items
                [...truncated code]

                // update the counter for next batch call
                $sandbox['current']++;
          }

          // Lets Batch APO know the percentage done
          // Used internally by Batch API
          // Batch won't stop until $sandbox['#finished'] is less than one
          // So this function will be call again and again until completion
          $sandbox['#finished'] = empty($sandbox['max']) ? 1 : ($sandbox['current'] / $sandbox['max']);

          // Once finished you can take some more actions
          if ($sandbox['#finished'] >= 1) {
                [... truncated code]
          }

         // Print message to batch api
         return t('Menu links converted');
    }
    else {
          // Print message to batch api
           return t('Menu link conversion skipped, because the {menu_tree} table did not exist yet.');
    }
}

Also have a look at Batch Example https://api.drupal.org/api/examples/batch_example%21batch_example.module/7.x-1.x

Answer 3

I've done this in a hook_update_N() hook. Pay attention to the &$sandbox array parameter in the update hook. What that allows you to do is to chunk off your users into byte sized bites and since it's passed by reference, it allows you to keep track of how many users you have processed.