Migrating a field collection with a field that has multiple values

Question

I have a field collection migration that is taking nodes of type Map List from a D6 site which have two fields, a title, and a node reference to entity of type Venue. There anywhere between 1-10 node reference fields on any given Map List.

The Map List nodes are to be migrated to a field collection on the D7 site called field_city_map_list which lives on the D7 nodes of type City. On the D6 site the Map List nodes were referenced on the old City content type via entity reference fields.

With the query and sourceMigrations set I was able to get the field collections to be attached to the correct City node in the D7 site, however in the case where a Map List has multiple node reference fields, the migration creates a single field collection for each node reference.

So where field_city_map_list should have several lists, with multiple node references like this:

CityNode->
  Title: Some Title
  City Map Lists: 
    Some List Title 1:
      345
      677
      987
    Some List Title 2:
      876
      678
      908
    Some List Title 3:
      493

It actually gets imported like this:

CityNode->
  Title: Some Title
  City Map Lists:
    Some List Title 1:
      345
    Some List Title 1:
      677
    Some List Title 1:
      987
    Some List Title 2:
      876
    Some List Title 2:
      678
    Some List Title 2:
      908
    Some List Title 3:
      493

I thought ordering the query by the map list nid and then by the delta in the field venue nid list would solve the issue but it didn't change the migration results at all. There are no errors or migration messages that appear.

Here is my class:

class FCMapList extends myFieldCollectionMigration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->systemOfRecord = Migration::DESTINATION;

    $this->description = t('Field Collection: Map List Nodes');

    $dependencies = array(
      'VenueProfilePage',
      'City',
    );

    $this->dependencies = $dependencies;

    $query = Database::getConnection('default', 'legacy')
      ->select('content_type_map_list', 'ml')
      ->fields('ml', array('nid', 'field_link_title_value'));
    $query->join('content_field_venues', 'v', 'v.nid=ml.nid');
    $query->fields('v', array('nid', 'field_venues_nid'));
    $query->join('content_field_lists', 'l', 'l.field_lists_nid=ml.nid');
    $query->fields('l', array('nid'));
    $query->orderBy('ml.nid');
    $query->orderBy('v.delta', 'ASC');
    $query->isNotNull('v.field_venues_nid');

    $this->source = new MigrateSourceSQL($query);

    $this->destination = new MigrateDestinationFieldCollection('field_city_map_list', array('host_entity_type' => 'node'));

    $this->addFieldMapping('host_entity_id', 'l_nid')
         ->sourceMigration('City');
    $this->addFieldMapping('field_list_title', 'field_link_title_value');
    $this->addFieldMapping('field_venues', 'field_venues_nid')
         ->sourceMigration('VenueProfilePage');

    $source_key = array(
      'nid' => array(
        'type' => 'int',
        'description' => 'List content ID',
        'alias' => 'ml'
      )
    );

    $this->map = new MigrateSQLMap($this->machineName, $source_key, MigrateDestinationFieldCollection::getKeySchema());

     }
}

I had found a similar question that was self-answered however as I said above, ordering by source node nid and field delta did not solve the issue.

I'm on migrate version: 7.x-2.6-rc1+57-dev and field collection version: 7.x-1.0-beta7

Answer 1

The only way around the issue I could find was to remove the map of D6 node reference field to the D7 entity reference field. Instead I set a default value to 1 so that it appears on the field collection in the prepare function, and I then repopulate it with a secondary query there. This solution was not ideal for me, but due to time constraints I was forced to go with this way. Hopefully someone can find a real solution to the issue.

One more issue I found while attempting to debug this behavior was that rolling back this field collection migration did not delete the field collections. It only reset the map messages and numbers of processed rows. To delete the field collections entirely I had to roll back the migration of the nodes they were on as well: drush migrate-rollback City && drush migrate-rollback FCMapList

class FCMapList extends myFieldCollectionMigration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->description = t('Field Collection: Map List Nodes');

    $dependencies = array(
      'VenueProfilePage',
      'City',
    );

    $this->dependencies = $dependencies;

    $query = Database::getConnection('mig_dest', 'legacy')
      ->select('content_type_map_list', 'ml')
      ->fields('ml', array('nid', 'field_link_title_value', 'field_link_subtitle_value'));
    $query->join('content_field_lists', 'l', 'l.field_lists_nid=ml.nid');
    $query->fields('l', array('nid'));
    $query->isNotNull('ml.nid');
    $query->isNotNull('l.field_lists_nid');

    $this->source = new MigrateSourceSQL($query);

    $this->destination = new MigrateDestinationFieldCollection('field_city_map_list', array('host_entity_type' => 'node'));

    $this->addFieldMapping('host_entity_id', 'l_nid')
         ->sourceMigration('City');
    $this->addFieldMapping('field_list_title', 'field_link_title_value');
    $this->addFieldMapping('field_venues')
     ->defaultValue(1);

    $source_key = array(
      'nid' => array(
        'type' => 'int',
        'description' => 'List content ID',
        'alias' => 'ml'
      )
    );

    $this->map = new MigrateSQLMap($this->machineName, $source_key, MigrateDestinationFieldCollection::getKeySchema());

     }

  public function prepare($field_collection, stdClass $row) {
    $legacy_list_nid = $row->nid;

    $venues_query = Database::getConnection('mig_dest', 'legacy')
      ->select('content_field_venues', 'v')
      ->fields('v', array('delta', 'field_venues_nid'))
      ->condition('v.nid', $legacy_list_nid, '=');
    $venues_query->orderBy('v.delta', 'ASC');
    $venues_query->orderBy('v.nid', 'ASC');
    $venues_query->isNotNull('v.field_venues_nid');
    $result = $venues_query->execute();

    while ($record = $result->fetchAssoc()) {
      $venue_delta = $record['delta'];
      $legacy_venue_nid = $record['field_venues_nid'];

      db_set_active();
      $venue_nid_query = Database::getConnection('mig_dest', 'default')
        ->select('migrate_map_venueprofilepage', 'mv')
        ->fields('mv', array('destid1'))
        ->condition('mv.sourceid1', $legacy_venue_nid, '=');
      $venue_result = $venue_nid_query->execute();
      $venue_nid = $venue_result->fetchField();

      if (is_array($venue_nid)) {
        $venue_nid = $venue_nid[0];
      }
      if (strlen($venue_nid) > 0) {
        $field_collection->field_venues['und'][$venue_delta]['target_id'] = (string)$venue_nid;
      }

    }

  }

}