1

I am writing a script to update all website registered user's Username & Email address. I have more then 5000+ users in my website which causes PHP execution time out error.

So I tried to implement Batch API but I am unable to implement it. I passed whole user list to Batch API and perform operations on it. But Batch API also shows the same PHP time out error.

Can someone help me to implement Batch API?

Here is my code:

function anonymize_data_menu() {
  $items = array();
  $items['admin/config/content/anonymize-user-data'] = array(
    'title' => 'Anonymize User Data',
    'description' => 'Click this link to anonymize user data',
    'type' => MENU_LOCAL_ACTION,
    'page callback' => '_anonymize_users_data',
    'access arguments' => array('administer user anonymize'),
    'file' => 'anonymize_data.user.inc',
  ); 
  return $items; 
}

function _anonymize_users_data() {
  $users = entity_load('user');
  $chunks = array_chunk($users, 20);
  $count_user = count($chunks);

  foreach ($chunks as $chunk) {
     $operations[] =  array('user_anonymize_op', array($chunk));
      //user_anonymize_op($chunk);
     //krumo($chunk);
  }
  $batch = array(
    'operations' => $operations,
    'title' => t('Import batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('An error occurred'),
    'finished' => 'user_anonymize_op_finished'
  );
  batch_set($batch);
  batch_process('admin/people');
}

function user_anonymize_op($users, &$context) {
  foreach ($users as $user) {
    # code...
    //krumo($user);
   $context['message'] = t('processing user @user', array('@user' => $user->name));
    //
      if($user->uid != 0 && $user->uid != 1) {
      $user->name = $i . '-' . $user->name;
      $user->pass = user_hash_password('blah');
      user_save($user);
    }
  }
}

function user_anonymize_op_finished($success, $results, $operations) {
  if ($success) {
    $message = count($results) . ' processed.';
  }
  drupal_set_message(t($message));
}

I am very confused about that how to process chunk of users.

Can someone help me it will be great.

  • In function user_anonymize_op, from if condition. – Pankaj Sachdeva Aug 26 '17 at 20:17
  • have you considered using set_time_limit (0) inside your script, I think the beginning of the user_anonymize_op() would be a good place and just before the foreach on _anonymize_users_data(). this is not a great solution, but might get you through the process. The solution is to work out why chunks of 20 people are taking so long it times out. also interested to know if there is any errors in the logs. – Andrew Killen Aug 30 '17 at 9:41
2

The timeout is caused because you're trying to load all 5K+ users before you start the batch. Instead get the User IDs first and load when processing. Check an example below. Also have a look at Batch API Overview documentation page, before you continue:

function anonymize_data_menu() {
  $items = array();
  $items['admin/config/content/anonymize-user-data'] = array(
    'title' => 'Anonymize User Data',
    'description' => 'Click this link to anonymize user data',
    'type' => MENU_LOCAL_ACTION,
    'page callback' => '_anonymize_users_data',
    'access arguments' => array('administer user anonymize'),
    'file' => 'anonymize_data.user.inc',
  ); 
  return $items; 
}

function _anonymize_users_data() {
  // Instead of loading all users at once,
  // first get there uid and then load chunk of user while processing.
  // $users = entity_load('user');
  $uids = db_select('users', 'u')
    ->condition('u.uid', 1, '>')
    ->fields('u', array('uid'))
    ->execute()->fetchAllKeyed(0, 0);

  $chunk_count = 20;
  // We've all rows here, split them in chunks.
  $chunks = array_chunk($uids, $chunk_count, TRUE);
  // Get all rows count.
  $num_operations = count($uids);

  $operations = array();
  foreach ($chunks as $chunk) {
    $operations[] =  array('user_anonymize_op', array(
      $chunk,
      array(
        'max' => $num_operations,
        'chunk' => $chunk_count,
      )
    ));
  }

  $batch = array(
    'operations' => $operations,
    'title' => t('Import batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('An error occurred'),
    'finished' => 'user_anonymize_op_finished'
  );

  batch_set($batch);
  batch_process('admin/people');
}

function user_anonymize_op($rows, &$context) {
  // Define variables on start of batch operation.
  if (!isset($context['results']['progress'])) {
    $context['results']['progress'] = 0;
    $context['results']['chunk'] = $batch_data['chunk'];
    $context['results']['max'] = $batch_data['max'];
    $context['results']['failed'] = array();
  }

  // Do your processing here. START HERE.
  $names = [];
  $users = entity_load('user', $rows);
  foreach ($users as $account) {
    if (!empty($account->uid) && ($account->uid > 1)) {
      // What is $i here?
      $account->name = $i . '-' . $account->name;
      $account->pass = user_hash_password('blah');
      user_save($account);
      $names[] = $account->name;
    }
  }
  // Do your processing here. END HERE.

  // Update completion count.
  $context['results']['progress'] = $context['results']['progress'] + count($rows);

  $context['message'] = t('Processed @name user data.', array('@name' => implode(', ', $names)));

  // Inform the batch engine that we are not finished,
  // and provide an estimation of the completion level we reached.
  if ($context['results']['progress'] != $context['results']['max']) {
    $context['finished'] = $context['results']['progress'] % $context['results']['max'];
  }
}

function user_anonymize_op_finished($success, $results, $operations) {
  if ($success) {
    $message = count($results) . ' processed.';
  }
  drupal_set_message(t($message));
}
0

First time it could seems to be a huge learning curve, but it isn't.

And once Batch API is learned, it is really powerfull and easy to implement.

Best way for starting, is looking at core example. The keyword you should look for is $sandbox. Which is the object that will be injected into each call of the batch. Often used on hook_update_n( $sandbox )

Here is a truncated example from system.install I commented out

I suggest to put your Batch into a hook_update and put your site in maintenance during the update process to prevent new user entry during the batch process, which could invalidate the batch somehow.

function system_update_8001(&$sandbox = NULL) {
      $database = \Drupal::database();
      $schema = $database->schema();

      // It checks if something has to be done
      if ($schema->tableExists('menu_tree')) {

            // If our sandbox object doesnt have the current property
            // It means this is the first call of the batch
            if (!isset($sandbox['current'])) {

                  [... truncated code ]

                  // So lets initiate the needed properties
                  $sandbox['current'] = 0;
                  $sandbox['max'] = $database->query('SELECT COUNT(mlid) FROM {menu_tree}')
                     ->fetchField();
           }

          // request the data from database
          // this is where $sandboc['current'] will be usefull 
           $menu_links = $database->queryRange('SELECT mlid, system_update_8001_title AS title, system_update_8001_description AS description FROM {menu_tree} ORDER BY mlid ASC', $sandbox['current'], $sandbox['current'] + 50)
             ->fetchAllAssoc('mlid');

          foreach ($menu_links as $menu_link) {

                // Process each items
                [...truncated code]

                // update the counter for next batch call
                $sandbox['current']++;
          }

          // Lets Batch APO know the percentage done
          // Used internally by Batch API
          // Batch won't stop until $sandbox['#finished'] is less than one
          // So this function will be call again and again until completion
          $sandbox['#finished'] = empty($sandbox['max']) ? 1 : ($sandbox['current'] / $sandbox['max']);

          // Once finished you can take some more actions
          if ($sandbox['#finished'] >= 1) {
                [... truncated code]
          }

         // Print message to batch api
         return t('Menu links converted');
    }
    else {
          // Print message to batch api
           return t('Menu link conversion skipped, because the {menu_tree} table did not exist yet.');
    }
}

Also have a look at Batch Example https://api.drupal.org/api/examples/batch_example%21batch_example.module/7.x-1.x

  • I didn't pay attention that your question was for Drupal 7. It works as the same, just look for core and contrib module that uses $sandbox in hook_update_n – SwingingTom Aug 31 '17 at 14:48
0

I've done this in a hook_update_N() hook. Pay attention to the &$sandbox array parameter in the update hook. What that allows you to do is to chunk off your users into byte sized bites and since it's passed by reference, it allows you to keep track of how many users you have processed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.