2

I'm running Drupal 8.3.6 and I try to import a json stream originating from typo3. It is essentially a list of blog posts in this format:

[{
    "id": "1",
    "title": "Title",
    "bodytext": "Bodytext with html Mark-Up",
    "datetime": "1269437640",
    "keywords": "term1, term2, term3, term4,..."
},{
    "id": "2",
    "title": "Title 2",
    "bodytext": "Another Bodytext with html Mark-Up",
    "keywords": "term6, term3, term9, term7,..."
}]

I came up with a small module following these instructions, which works well for the title and bodytext:

protected function createBlog(string $json) {
    $jsonout = json_decode($json, TRUE);

    foreach ($jsonout as $blog) {

      $node = Node::create(array(
        'type' => 'blog',
        'langcode' => 'de',
        'uid' => '1',
        'status' => 1,
        'title' => $blog['title'],
        'body' => ['value' => $blog['bodytext'], 'format' => 'full_html'],
      ));

      $node->save();
    }
  }

However I'm struggling to get the terms created and referenced by the nodes. I tried something like this, but it doesn't work:

protected function createBlog(string $json) {
    $jsonout = json_decode($json, TRUE);

    foreach ($jsonout as $blog) {
      foreach ($blog['keywords'] as $tag) { // Loop through each blog entries keywords and create terms
        $term = Term::create(array(
          'name' => $tag,
          'vid' => 'tags',
        ))->save();
      }

      $node = Node::create(array(
        'type' => 'blog',
        'langcode' => 'de',
        'uid' => '1',
        'status' => 1,
        'title' => $blog['title'],
        'field_tags' => $blog['keywords'], // I would need to have the terms from above in an array to add them here, right?
        'body' => ['value' => $blog['bodytext'], 'format' => 'full_html'],
      ));

      $node->save();
    }
  }

The terms do not get created and if they would, I wouldn't know how to associate them with the nodes. Also if it would work, there would be the problem that on creation of the terms I do not check if a term already exists. This would be necessary because nodes can have the same terms.

Is it possible to achieve this with my approach? If yes, can you tell me how? I also looked in to migrate_plus but I find it quite complicated.

Any help would be appreciated!

Andi

  • tl;dr. create migration. – user21641 Aug 10 '17 at 13:18
  • I recommend using the Migrate API (drupal.org/docs/8/api/migrate-api/migrate-api-overview) and the migrate_plus module (drupal.org/project/migrate_plus) which provides a URL plugin, a JSON source, and an entity lookup process plugin that will look up taxonomy terms and store their ID with the node you're creating. This blog entry (ohthehugemanatee.org/blog/2017/06/07/…) was instrumental in helping me quickly grasp migrations. – mounty Aug 10 '17 at 13:25
  • @mounty Thanks for the link! I didn't come across before. It seems, that is exactly what I need to get it started! – andir Aug 11 '17 at 11:46
  • I posted my answer below. – andir Aug 11 '17 at 13:42
1

Not knowing how the rest of your dependencies are set up, one thing I notice off the bat is that your keywords source field is a comma-separated single string, but you're treating it as an array of values in your processor. Maybe something like this:

protected function createBlog(string $json)
{
    $jsonout = json_decode($json, TRUE);

    foreach ($jsonout as $blog) {
        $newTags = explode(', ', $blog['keywords']);
        $newTagIDs = [];
        foreach ($newTags as $tag) {
            $newTagIDs[] = Term::create(array(
                'name' => $tag,
                'vid' => 'tags'
            ))->save();
        }

        $node = Node::create(array(
            'type' => 'blog',
            'langcode' => 'de',
            'uid' => '1',
            'status' => 1,
            'title' => $blog['title'],
            'field_tags' => $newTagIDs,
            'body' => ['value' => $blog['bodytext'], 'format' => 'full_html']
        ));

        $node->save();
    }
}

You're right that your approach would create multiple copies of the same term, though. The Entity Lookup plugin from the migrate_plus module might provide some help:

// $this->entityManager is a dependency-injected instance of Drupal\Core\Entity\EntityManagerInterface
$query = $this->entityManager->getStorage('taxonomy_term')
    ->getQuery()
    ->condition('name', $termNameToLookup)
    ->condition('vid', $vocabularyToLookup);
$results = $query->execute();
if (empty($results)) {
    // term doesn't exist in that vocabulary; should add it
} else {
    // the term already exists in that vocabulary; do something with it
}
  • Thanks for your help! However, I couldn't find out how to add multiple terms on node creation. 'field_tags' => $newTagIDs, does not work. For a specific term it is: 'field_tags' => [ ['target_id' => 277] ], Also, I couldn't figure out how to check if a term already exists. I will digg into migrate_plus for a bit now. – andir Aug 11 '17 at 9:47
  • Looks like you solved it via migration, but in the interest of not leaving this unanswered: I've actually never added multiple terms when using Node::create() so I assumed it would be just adding a flat array of tids. You could probably modify the for loop to create an array of arrays: $newTagIDs[] = ['target_id' => Term::create(...)->save()]; But the migration is definitely the way to go here. :) – mounty Aug 12 '17 at 16:28
0

Thanks to mounty for pointing me to the tutorial of Campbell Vertesi to get me started with d8 migrate: https://ohthehugemanatee.org/blog/2017/06/07/stop-waiting-for-feeds-module-how-to-import-remote-feeds-in-drupal-8/

It's a blessing. Also the d8 docs are useful: https://www.drupal.org/docs/8/api/migrate-api/migrate-process-plugins/migrate-process-overview

The crucial part to get my list of tags migrated is this pipeline:

field_tags:
    -
      plugin: explode
      source: keywords
      delimiter: ','
    -
      plugin: entity_generate
      lookupBundleKey: vid
      lookupBundle: tags

The explode plugin grabs the tags from the list and the entity_generate plugin checks if the entity exists. If not, the entity is created. The tags are then added to the node. The whole yml file for my import:

id: blog_importer
label: 'Blog Importer'
status: true

source:
  plugin: url
  data_fetcher_plugin: http
  urls: 'url/file.json'
  data_parser_plugin: json

  item_selector: blog-entry
  fields:
    -
      name: bid
      label: 'Blog Post ID'
      selector: uid
    -
      name: title
      label: Title
      selector: title
    -
      name: body
      label: Body
      selector: bodytext
    -
      name: keywords
      label: Keywords
      selector: keywords

  ids:
    bid:
      type: integer

destination:
  plugin: 'entity:node'

process:
  title: title
  'body/format':
    plugin: default_value
    default_value: "full_html"
  'body/value': body
  field_tags:
    -
      plugin: explode
      source: keywords
      delimiter: ','
    -
      plugin: entity_generate
      lookupBundleKey: vid
      lookupBundle: tags
  status:
    plugin: default_value
    default_value: 1
  type:
    plugin: default_value
    default_value: toppik_blog

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.