0

I'm learning how to create migrations in D8. However, I have this annoying issue where my migration is listed when I execute drush migrate-status but the total and unprocessed values are shown as N/A, shown below:

Group: JSON People (people)  Status  Total  Imported  Unprocessed  Last imported
 nodetest                     Idle    N/A    0         N/A
 people_data                  Idle    1000   0         1000

My json config is as follows:

id: nodetest
description:  Testing for a node migration
label:  Node Test
migration_group: people

source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  urls:
    - 'http://api.tvmaze.com/search/shows?q=family'
  # path: 'http://api.tvmaze.com/search/shows?q=family'
  item_selector:  /
  # identifierDepth: 1
  ids:
    show:
      id:
        type: integer
  fields:
    -
      name: name
      label: Name of show
      selector: name

process:
  type:
    plugin: default_value
    default_value: page
  title: name

destination:
  plugin: 'entity:node'

I have been unable to figure out what the issue is some I get figures rather than the N/A output I'm currently getting.

Any help, please!

UPDATE: I found I was incorrectly using show as the item_selector. Infact, the json from the api service does not have a "label" which refers to all the json objects, as shown below: the first of 10 json objects

So, I remove item_selector from the yml data and I then get 10 instead of N/A in the total column of the nodetest migration.

The problem now is I get Invalid argument supplied for foreach() DataParserPluginBase.php:121 [warning] warnings (1 for each json object).

And, when I attempt to migrate, I get the following error:

Invalid argument supplied for foreach() DataParserPluginBase.php:121                                                                                                      [warning]
Invalid argument supplied for foreach() DataParserPluginBase.php:121                                                                                                      [warning]
TypeError: Argument 2 passed to Drupal\migrate\Row::__construct() must be of the type array, null given, called in                                                        [error]
/var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php on line 357 in Drupal\migrate\Row->__construct() (line 101 of
/var/www/d8/core/modules/migrate/src/Row.php) #0 /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(357):
Drupal\migrate\Row->__construct(Array, NULL)
#1 /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(330): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->next()
#2 [internal function]: Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#3 /var/www/d8/core/modules/migrate/src/MigrateExecutable.php(188): FilterIterator->rewind()
#4 /usr/local/share/drush/includes/drush.inc(721): Drupal\migrate\MigrateExecutable->import()
#5 /usr/local/share/drush/includes/drush.inc(712): drush_call_user_func_array(Array, Array)
#6 /var/www/d8/modules/contrib/migrate_tools/migrate_tools.drush.inc(317): drush_op(Array)
#7 [internal function]: _drush_migrate_tools_execute_migration(Object(Drupal\migrate\Plugin\Migration), 'nodetest', Array)
#8 /var/www/d8/modules/contrib/migrate_tools/migrate_tools.drush.inc(272): array_walk(Array, '_drush_migrate_...', Array)
#9 /usr/local/share/drush/includes/command.inc(422): drush_migrate_tools_migrate_import('nodetest')
#10 /usr/local/share/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#11 /usr/local/share/drush/includes/command.inc(199): drush_command('nodetest')
#12 /usr/local/share/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#13 /usr/local/share/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#14 /usr/local/share/drush/drush.php(12): drush_main()
#15 {main}.
TypeError: Argument 2 passed to Drupal\migrate\Row::__construct() must be of the type array, null given, called in /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php on line 357 in /var/www/d8/core/modules/migrate/src/Row.php on line 101 #0 /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(357): Drupal\migrate\Row->__construct(Array, NULL)
#1 /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(330): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->next()
#2 [internal function]: Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#3 /var/www/d8/core/modules/migrate/src/MigrateExecutable.php(188): FilterIterator->rewind()
#4 /usr/local/share/drush/includes/drush.inc(721): Drupal\migrate\MigrateExecutable->import()
#5 /usr/local/share/drush/includes/drush.inc(712): drush_call_user_func_array(Array, Array)
#6 /var/www/d8/modules/contrib/migrate_tools/migrate_tools.drush.inc(317): drush_op(Array)
#7 [internal function]: _drush_migrate_tools_execute_migration(Object(Drupal\migrate\Plugin\Migration), 'nodetest', Array)
#8 /var/www/d8/modules/contrib/migrate_tools/migrate_tools.drush.inc(272): array_walk(Array, '_drush_migrate_...', Array)
#9 /usr/local/share/drush/includes/command.inc(422): drush_migrate_tools_migrate_import('nodetest')
#10 /usr/local/share/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#11 /usr/local/share/drush/includes/command.inc(199): drush_command('nodetest')
#12 /usr/local/share/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#13 /usr/local/share/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#14 /usr/local/share/drush/drush.php(12): drush_main()
#15 {main}
TypeError: Argument 2 passed to Drupal\migrate\Row::__construct() must be of the type array, null given, called in /var/www/d8/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php on line 357 in Drupal\migrate\Row->__construct() (line 101 of /var/www/d8/core/modules/migrate/src/Row.php).
Drush command terminated abnormally due to an unrecoverable error.
Improve this question
10
  • I'm not 100% certain, but can you verify the item_selector is correct and returning items?
    – Kevin
    Commented Jun 26, 2019 at 1:45
  • @Kevin: I wondered about that but it seems correct to me. Please visit the source:url to confirm.
    – sisko
    Commented Jun 26, 2019 at 8:51
  • @Kevin: After further consideration, I think you were right about the item_selector. I updated my question
    – sisko
    Commented Jun 26, 2019 at 11:07
  • I see what you mean, the results themselves have no wrapper to touch. Did you try just '/' for item selector?
    – Kevin
    Commented Jun 26, 2019 at 13:19
  • @Kevin: I had tried everything but /. I have now tried tried it, I am getting 10 items BUT the migration fails with the following message: Migration failed with source plugin exception: id is defined as a source ID but has no value. [error]
    – sisko
    Commented Jun 26, 2019 at 14:13

1 Answer 1

Reset to default
1

The source in your migration yaml should look more like this:

source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  urls:
    - 'http://api.tvmaze.com/search/shows?q=family'
  item_selector: /
  fields:
    -
      name: id
      label: "Field you want to be the source id"
      selector: path/to/id
    -
      name: name
      label: "Name of show"
      selector: name
  ids:
    id:
      type: integer

ids are picked from fields. In the yaml provided above, you are missing the ids definition and its field.

Improve this answer
1
  • Thanks dude! That was extremely helpful
    – sisko
    Commented Jun 26, 2019 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.