How can I fix "Allowed memory size exhausted" on batch finish?

Question

The batch processed all the items, but instead of showing the finish message, I see the error "Allowed memory size of 536870912 bytes exhausted".

When I debug the code, I notice that Drupal loads each proccessed block on batch finish (ContentEntityBase->__construct). I can't figure out why Drupal is doing that.

Structure of the code:

This is a custom module with form class, and batch functions in the custom_module.module file. On form submit, the module calls the create batch function:

public function submitForm(array &$form, FormStateInterface $form_state) {
  custom_module_make_batch();
}

The custom_module_build_batch function getting ids of custom blocks (4000 or more) and generating the batch:

function custom_module_make_batch()
{

  $batch = [];

  $items = get_blocks_ids();

  $batch = custom_module_generate_batch($items);
  batch_set($batch);
}

function custom_module_generate_batch($items)
{

  $operations = [];

  $operations_groups = array_chunk($items, 50);

  foreach ($operations_groups as $key => $operations_group) {
    $operations[] = [
      'custom_module_batch_op',
      [$operations_group],
    ];
  }

  $batch = [
    'operations' => $operations,
    'finished' => 'custom_module_batch_finished',
    'title' => 'Custom batch',
    'init_message' => 'Batch is starting.',
    'progress_message' => 'Processed @current out of @total parts.',
    'error_message' => 'Batch has encountered an error.',
  ];
  return $batch;
}

function custom_module_batch_op($operations_group, &$context) {

  foreach ($operations_group as $key => $bid) {
  
    $block = \Drupal::service('entity.repository')->loadEntityByUuid('block_content', $bid);

    $block->field_name = $new_value;
    
    $block->save();
  }

}

function custom_module_batch_finished($success, $results, $operations)
{

  $messenger = \Drupal::messenger();
  if ($success) {
    // Here we could do something meaningful with the results.
    // We just display the number of nodes we processed...

    if ($total) {
      $messenger->addMessage(t('@count results processed.', ['@count' => $total]));
    } else {
      $messenger->addMessage(t('There no items for the migration'));
    }

  } else {
    // An error occurred.
    // $operations contains the operations that remained unprocessed.
    $error_operation = reset($operations);
    $messenger->addMessage(
      t(
        'An error occurred while processing @operation with arguments : @args',
        [
          '@operation' => $error_operation[0],
          '@args' => print_r($error_operation[0], TRUE),
        ]
      )
    );
  }
}

Answer 1

Batch operations are used when the number of items to handle is unknown and there could possibly be so much items that either loading all of them would use all the memory PHP make available, or handling them would take more time than the time PHP gives to the script to run. When data is loaded from a database table, even entity data, it's the batch operation callback that loads the data, like this code does, for example.

$batch_builder = (new BatchBuilder())
  ->setTitle(t('Deleting database rows'))
  ->setFinishCallback('mymodule_finished_callback')
  ->addOperation('mymodule_delete_rows', []);

batch_set($batch_builder->toArray());

function mymodule_delete_rows(&$context) {
  $connection = \Drupal::database();

  if (empty($context['sandbox'])) {
    $context['sandbox']['progress'] = 0;
    $context['sandbox']['current_id'] = 0;
    $context['sandbox']['max'] = $connection
      ->query('SELECT COUNT(DISTINCT [id]) FROM {example}')
      ->fetchField();
    $context['results'] = 0;
  }

  $limit = 5;
  $result = $connection
    ->select('example')
    ->condition('id', $context['sandbox']['current_id'], '>')
    ->orderBy('id')
    ->range(0, $limit)
    ->execute();

  foreach ($result as $row) {
    $context['sandbox']['progress']++;
    $context['results']++;
    $context['sandbox']['current_id'] = $row->id;
    
    if (!empty($row->title) && is_numeric($row->title[0])) {
      $connection->delete('example')
        ->condition('id', $row->id)
        ->execute()
    }
  }
  if ($context['sandbox']['progress'] != $context['sandbox']['max']) {
    $context['finished'] = $context['sandbox']['progress'] / $context['sandbox']['max'];
  }
}

function mymodule_finished_callback($success, $results, $operations, $elapsed) {
  if ($success) {
    $message = \Drupal::translation()->formatPlural($results, '@count row deleted.', '@count rows deleted.');
  }
  else {
    $message = t('Finished with an error.');
  }
  \Drupal::messenger()->addMessage($message);
}

The relevant values set in the batch operations callback are the following ones:

• $context['finished'] tells Drupal to stop calling the batch operations callback, when its value is 1 or $context['finished'] isn't set
• $context['results'] is passed to the callback invoked when the batch operations are done, as its second parameter
• $context['message'] is a text message displayed in the progress page

I would not build an array containing a value for each handled item, as that would use all the available memory, when the handled items are much.

Reference

• Batch operations
• callback_batch_operation()
• callback_batch_finished()
• BatchBuilder