0

I'm wondering what could be causing my migration to run so slowly. I'm using dev versions of migrate and d2d as I always have without issue. There are over 120,000 users and the first 2000-9000 import within a few seconds.

After that it only processes 2 at a time (about every 30 seconds).

d2d code:

  $common_arguments = array(
    'source_connection' => 'legacy',
    'source_version' => 6,
  );

  $arguments = $common_arguments + array(
    'description' => t('Migration of users from Drupal 6'),
    'machine_name' => 'User',
  );

  Migration::registerMigration('DrupalUser6Migration', $arguments['machine_name'], $arguments);

include file:

class User extends DrupalUser6Migration {
  public function __construct(array $arguemnts) {
    parent::__construct($arguemnts);

    $this->addFieldMapping('roles', 'roles')
      ->sourceMigration('Roles');
  }
}

The legacy database the data is coming from is on my local machine on the same SSD the new site is on. Any ideas on why this is running so slow? At the rate it's going out it will take over a month to import all of the users...

enter image description here

1

In settings.php set:

$conf['password_count_log2'] = 1;

You will speed up how drupal handles user passwords: The problems reported here: https://www.drupal.org/node/2200181 ... seem to match yours, first 2000 in X seconds second 2000 in X.Y seconds etc...

Also, in php.ini set:

max_execution_time to 0 (for unlimited) or a high number like 3000 rather than the default 30.

NB. Be sure to return the execution time to its default setting after the migration.

  • After doing this it is going slightly faster. Instead of only doing 2 at a time it will process 2 at a time for about 5min, then process about 100, then go back to 2 at a time for another 5 or so minutes. – CR47 Jun 30 '14 at 17:03
  • When I do as the guy in the article you linked did and limit to 1000 at a time, I still get 2 people importing every 30seconds – CR47 Jun 30 '14 at 17:06
  • 1
    Ok do you have pathauto turned on? If yes turn it off.. – iain maitland Jun 30 '14 at 17:09
  • No it is disabled. I never had it enabled. – CR47 Jun 30 '14 at 17:14
  • 1
    ok bump that up to 3000 and see if it makes a difference. – iain maitland Jun 30 '14 at 17:27
0

CR47. I had same problem, I had to migrate 300 000 of users and batch migration became very slow. I used drush migrate commands and migration was rather fast (approximately 1000 of users per minute). I used

drush migrate-import User

where User is a machine name of my migration.

Also you can use

drush migrate-import User --feedback="1000 items"

In this case you will get feedback when 1000 of users will be migrated.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.