1

I need to migrate a json feed with following example structure;


{  
   "data":[  
      {  
         "id":"11028",
         "first_name":"Foo",
         "last_name":"Bar",
         "functions":{  
            "group_function":{  
               "data":[  
                  {  
                     "group":"1",
                     "function":"Scrummaster",
                     "active":"1"
                  },
                  {  
                     "group":"2",
                     "function":"Developer",
                     "active":"0"
                  }
               ]
            }
         }
      },
      {  
         "id":"11029",
         "first_name":"John",
         "last_name":"Doe",
         "functions":{  
            "group_function":{  
               "data":[  
                  {  
                     "group":"1",
                     "function":"Project manager",
                     "active":"1"
                  },
                  {  
                     "group":"2",
                     "function":"Project owner",
                     "active":"1"
                  }
               ]
            }
         }
      }
   ]
}

For this, I was thinking of having a teammember node, and a teamfunction paragraph type for the functions for each team member.

If i understand migrate api correctly, i create 2 migrations for this. One for the node and one for the paragraphs.

How can I iterate over the group_function/data array within my paragraphs migration?

I tried:


source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  headers:
    Accept: 'application/json; charset=utf-8'
    Content-Type: 'application/json'
  urls: example.com/team.json
  item_selector: data/functions

But I get following response on a 'drush ms':
Could not retrieve source count from team_function: Passed variable is not an array or object

How would you do this?

2

You could also create a process plugin to create the Paragraph inline with the record you are migrating. I can't do piecemeal migrations in my case. The project has about 100,000 nodes and no data structure to 'create' Paragraphs out of. Even if I could, I don't want to have to run that dozens of times to get a full set of data.

Anyway.

Here is a real example:

<?php

namespace Drupal\mymodule_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;
use Drupal\paragraphs\Entity\Paragraph;

/**
 * @MigrateProcessPlugin(
 *   id = "published_author_paragraph_item",
 *   handle_multiples = TRUE
 * )
 */
class PublishedAuthorParagraphItem extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $paragraphs = [];

    foreach ($value as $item) {
      $paragraphs[] = $this->createParagraphItem($item);
    }

    return $paragraphs;
  }

  /**
   * {@inheritdoc}
   */
  public function multiple() : bool {
    return TRUE;
  }

  /**
   * Generates a paragraph item entity.
   *
   * @param array $value
   * @return array
   * @throws \Drupal\Core\Entity\EntityStorageException
   * @throws \Drupal\migrate\MigrateException
   */
  protected function createParagraphItem(array $value) : array {
    $paragraph = Paragraph::create([
      'type' => 'paper_authors',
      'field_paper_author' => [
        'target_id'  =>  $value['user_id'],
      ],
      'field_paper_published_name' => [
        'value'  =>  $value['name'],
      ],
      'field_paper_disclosure' => [
        'value'  =>  $value['disclosure'],
        'format' => 'basic_html',
      ],
    ]);

    $paragraph->save();

    return [
      'target_id' => $paragraph->id(),
      'target_revision_id' => $paragraph->getRevisionId(),
    ];
  }
}

In my migration definition yaml:

  _authors_shim:
    plugin: generate_authors_array
    source: author_array
  field_paper_published_authors:
    plugin: published_author_paragraph_item
    source: '@_authors_shim'

generate_authors_array is another process plugin I use to massage the data for the plugin code above. This paragraph type has a user reference, a plain text field and long text field. The passed value is an array of arrays, so this will in fact create multiple paragraph deltas for the paragraph field I am filling. It also rolls back properly.

I had tried to get this to work with sub_process at first, but could not get the paragraph field to populate at all until I went this route. I believe this is due to paragraph fields hold references to their stored entities (target_id, target_revision_id).

  • thanks, wrote a migrate plugin based on your code and got it running. Writing your own plugin is indeed way easier. :) – betz Mar 31 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.