In my Drupal 6 site I have 147490 files in the database. On my Drupal 7 site I am using D2D migrate module to import them. The number of Items it sees is exactly the same, 147490. However, it will not processes 5683 of them. 112229 import successfully and 29578 fail (because the file no longer exists on the server, that's fine). When re-running the migration, it will not processes the remaining 5683.
Here is my migration class:
$file_arguments = $common_arguments + array(
'machine_name' => 'Files',
'description' => t('Migration of files from Drupal 6'),
'user_migration' => 'User',
'default_uid' => 1,
'source_dir' => 'http://lcl.legacy-d6.com/sites/default/files',
'destination_dir' => 'public://',
);
Migration::registerMigration('Files', $file_arguments['machine_name'], $file_arguments);
I am then extending the class with an include: file_migration.inc
class Files extends DrupalFile6Migration {
public function __construct(array $arguemnts) {
parent::__construct($arguemnts);
}
public function prepareRow($row) {
if (parent::prepareRow($row) === FALSE) {
return FALSE;
}
$filepath = $row->filepath;
$new_filepath = str_replace('/var/www/html/liv-dir/production/', '', $filepath);
$new_filepath = str_replace('/var/www/html/lcl.legacy-d6.com/htdocs/', '', $new_filepath);
$row->filepath = $new_filepath;
return TRUE;
}
}
EDIT: I was able to create a snippet that finds all of the missing Ids however I was not able to figure out why they are not being processed.
set_time_limit(0);
$unprocessed_ids = array();
db_set_active('legacy');
$get_fids_query = Database::getConnection('legacy')->select('files', 'fi');
$get_fids_query->fields('fi', array('fid'));
$result = $get_fids_query->execute();
while ($fid = $result->fetchField()) {
db_set_active();
$check_migration_query = Database::getConnection('default')->select('migrate_map_files', 'mfi');
$check_migration_query->fields('mfi', array('sourceid1'));
$check_migration_query->condition('sourceid1', $fid, '=');
$check_migration_result = $check_migration_query->execute();
$check_migration = $check_migration_result->fetchField();
if ($check_migration === FALSE || is_null($check_migration) || !isset($check_migration)) {
$unprocessed_ids[] = $fid;
}
db_set_active('legacy');
}
}
UPDATE: The only fix I could find that actually worked was waiting for the migration to run its course and then rollback a small number of rows, and then run through the migration again with --update flag set. Depending on the envioronment I was running on (local vs pantheon) I would have to do this between 5-20 times.
I still want to know why this was happening.
drush mmsg Files
say?