Migrating paragraph items to their host nodes from Drupal 7 to Drupal 9?

Question

I am trying to do a Drupal 7 -> Drupal 9 migration using the UI. It was able to convert field collection types to paragraph types. I was also able to create a few custom migrations using the d7_paragraphs_item plugin provided by Paragraphs to migrate all paragraph items from D7 into the database.

Now I need to attach them to the nodes.

My migration yaml:

id: field_content
migration_tags: {}
migration_dependencies: {}
migration_group: yale
label: Field Content Paragraphs
source:
  plugin: node_items
  key: migrate
destination:
  plugin: 'entity:node'
process:
  nid:
    - plugin: migration_lookup
      migration: d7_node_complete:page
      source: entity_id
      no_stub: true
    - plugin: extract
      index:
        - 0
  title: title
  items:
    plugin: migration_lookup
    migration:
      - accordion_items_paragraph
      - editor_items
      - highlight_files
    source: field_content_value
  field_content:
    plugin: sub_process
    source:
      - '@items'
    process:
      target_id: '0'
      target_revision_id: '1'

Except this is only doing maybe 20% of the total. It would appear that its not pulling back all possible paragraph items to stick into the field, or maybe I am just thinking of it wrong.

The source plugin is returning a set of data from Drupal 7 for a field that held Paragraph references:

class NodeItems extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('field_data_field_content', 'fc');
    $query->join('node', 'n', 'n.nid = fc.entity_id');
    $query->join('paragraphs_item', 'pi', 'pi.item_id = fc.field_content_value');
    $query->fields('fc', [
        'entity_id',
        'bundle',
        'delta',
        'field_content_value'
      ]);
    $query->fields('n', [
      'title',
    ]);
    $query->condition('n.status', 1);
    $query->condition('fc.bundle', 'accordions', '!=');
    $query->orderBy('fc.entity_id');
    $query->orderBy('fc.delta');
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'title' => $this->t('title'),
      'entity_id' => $this->t('entity_id'),
      'bundle' => $this->t('bundle'),
      'delta' => $this->t('delta'),
      'field_content_value' => $this->t('field_content_value'),
    ];
  }

"field_content" in D9 is a Paragraphs field. It does seem to plug some values, I do see items on various nodes. But its failing because I think the lookup doesn't always return a value or maybe it doesn't know what to do with them? Also the nid: mapping needs to support multiple node types, I passed one migration. There are 12. Can I add all of them there or ?

Is this the way to attach them to host nodes? Is there a better way?

Answer 1

I was thinking about this in the wrong way. The source plugin returns a list of results to migrate one at a time - the SQL query I was using would cause you to get for example, 5 rows for one node, each with a paragraph id. That is why only one paragraph was showing up on nodes at the end of a migration.

That isn't what you want - what you want instead is one row with the node id and 5 paragraph IDs.

I was able to do that by adjusting my source plugin query to the following:

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('node', 'n');
    $query->join('D7_FIELD_NAME', 'fc', 'fc.entity_id = n.nid');
    $query->fields('n', [
      'nid',
      'title'
    ]);
    $query->condition('n.status', 1);
    $query->addExpression('GROUP_CONCAT(fc.D7_FIELD_NAME_VALUE)', 'paragraph_ids');
    $query->groupBy('n.nid');
    $query->groupBy('n.title');
    $query->orderBy('fc.delta');
    return $query;
  }

This produces a result set of

nid | paragraph_ids
-------------------
1     2, 4, 6, 8
2     13, 18, 12, 9
...........

Once you have that part, then you can fill in your Migration definition:

  D9_DESTINATION_FIELD_NAME:
    - plugin: explode
      delimiter: ','
      source: paragraph_ids
    - plugin: to_array
    - plugin: sub_process
      process:
        target_id:
          - plugin: migration_lookup
            migration:
              - accordion_items_paragraph
              - editor_items
              - highlight_files
            source: value
          - plugin: extract
            index:
              - 0
        target_revision_id:
          - plugin: migration_lookup
            migration:
              - accordion_items_paragraph
              - editor_items
              - highlight_files
            source: value
          - plugin: extract
            index:
              - 1

explode changes the comma delimited list into an array, and to_array is a small plugin I made to make it an array of arrays, each with an index of "value" and the id.

sub_process iterates over that and does a lookup against my other 3 paragraph migrations to drop the new destination id into target_id and target_revision_id.

The nid mapping from the original question still works, and passing multiple node migrations provides the full lookup capability. So for any nodes the result set returns, they are updated accordingly.

Now all migrated Paragraphs show up on their respective nodes in the correct order. I was able to apply this pattern to 3 different entity_reference_revision fields in Drupal 9 that held Paragraphs.