How to migrate using explode with occasional empty array

Question

We've made a migration from the City of Helsinki's Linked Events API, and during this work we've been seeing these migration messages:

Array index missing, extraction failed

Here is an example of an event that works nicely: https://api.hel.fi/linkedevents/v1/event/hko:14004/

And one that causes the error: https://api.hel.fi/linkedevents/v1/event/matko:16134/

In the latter you see that offers is returned as an empty array. We're using the skip_on_empty plugin to handle those:

field_offers_is_free:
  -
    plugin: skip_on_empty
    method: process
    source: offers
  -
    plugin: extract
    index:
      - 0
      - is_free
field_offers_info_url:
  -
    plugin: skip_on_empty
    method: process
    source: offers
  -
    plugin: extract
    default: false
    index:
      - 0
      - info_url
      - fi

The problem is that never reaches the skip_on_empty plugin. I've been stepping through this in the debugger based on my findings I created an issue in Drupal Core https://www.drupal.org/node/2905929

When Get sets $multiple to true, processRow() in MigrateExecutable runs a foreach () on an empty array - resulting in the skip_on_empty plugin to be skipped.

Is there something wrong in our setup or did I uncover a bug? Any ideas on how to work smoothly around this?

Answer 1

I solved this by creating my own migrate plugin that sets the multiple flag to false if we get an empty array, like this:

/**
 * The Get plugin causes issues for us when it fetches and array that is empty.
 *
 * It sets multiple to true which prevents the skip_on_empty plugin from being
 * triggered.
 * @see https://www.drupal.org/node/2905929
 *
 * @MigrateProcessPlugin(
 *   id = "get_array_fix_multiple",
 *   handle_multiples = TRUE
 * )
 */

class GetArrayFixMultiple extends ProcessPluginBase {

  /**
   * Flag indicating whether there are multiple values.
   *
   * @var bool
   */
  protected $multiple;

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
      $array = is_array($value);
      $empty = empty($value);
      $this->multiple = $array && !$empty;
      return $value;
  }

  /**
   * {@inheritdoc}
   */
  public function multiple() {
    return $this->multiple;
  }

}

Either the bug is in the get plugin or then skip_on_empty should support multiple.