1

I'm trying to migrate from Google Sheets into Drupal 8.9+. I have successfully configured most of the migration, but am having trouble migrating two source link fields (Event URL & Ticketing URL) into ONE multi-value links field (field_links) in my target node. I'm trying to shape all the source data in my Google Sheet so I don't have to use custom process plugins.

Here's a link to an example of the Google Spreadsheet I'm using as the source.

I found this Github Gist that led me to believe I could shape the two url fields I have in Google sheets into an array, and have that array be processed by the iterator (now, sub_process) plugin. So I now have a source column called "LinksArray" that merges the data from my "Event URL" and "Ticking URL" columns to create an array that looks exactly like this:

[
  ['title' => 'Event URL', 'uri' => 'https://coolvenue.com/etc'],
  ['title' => 'Ticketing URL', 'uri' => 'https://tickets.com/etc'],
];

And the relevant migration file looks like this:

...

    - name: src_links_array
      label: "LinksArray"
      selector: linksarray
...

  field_links:
    - plugin: skip_on_empty
      method: process
      source: src_links_array
      message: 'field_links is empty. Skipping.'
    - plugin: sub_process
      process:
        title: title
        uri: uri

When I run the migration, It appears to be successful, with no errors, but the content created is missing the links altogether (no titles, no uri's, nothing), so is obviously not working. I tried reformatting the array with and without outer brackets, with and without trailing commas, etc. Nothing works. Is my manually created array being written incorrectly? Do I need to create the array inside the migration instead? If so, I'd greatly appreciate help with that. Thank you!

2

The two main ways to approach this with php are via hook or plugin.

The first is with a hook, specifically, hook_migrate_MIGRATION_ID_prepare_row().

Allows adding data to a row for a migration with the specified ID.

For example, you could combine your two link columns to a new column formatted as an array.

function my_module_migrate_MY_MIGRATION_ID_prepare_row(Row $row, MigrateSourceInterface $source, MigrationInterface $migration) {
  // Assuming your url fields are keyed event_url and ticketing_url in your migration.yml.
  $links_array = [];
  if ($row->hasSourceProperty('event_url')) {
    $event_url = $row->getSourceProperty('event_url');
    if (!empty($event_url)) {
      $links_array[] = [
        'title' => 'Event URL',
        'uri' => $event_url,
      ];
    }
  }
  if ($row->hasSourceProperty('ticketing_url')) {
    $ticketing_url = $row->getSourceProperty('ticketing_url');
    if (!empty($ticketing_url)) {
      $links_array[] = [
        'title' => 'Ticketing URL',
        'uri' => $ticketing_url,
      ];
    }
  }

  $row->setSourceProperty('event_and_ticketing_url', $links_array);
}

You can use your programmatically added source value in your process section like:

process:
  ...
  field_links: event_and_ticketing_url
  ...

I prefer to use plugins over hooks whenever possible for a few reasons, but one is because plugins are more visible to anybody looking at the implementation later. By looking at the migration.yml, a future developer will know that a custom plugin does something to this thing and the plugin can be found in the src/Plugin directory.

An example doing the same thing, but with a plugin might look like:

my_module/src/Plugins/migrate/process/EventAndTicketingLinks.php:

/**
 * Process plugin to combine event and ticketing url sources to one links array.
 *
 * Example Usage:
 *
 * field_my_links_field:
 *   plugin: my_module_event_and_ticketing_links
 *   link_fields:
 *     event: event_url
 *     ticketing: ticketing_url
 *
 * @MigrateProcessPlugin(
 *   id = "my_module_event_and_ticketing_links",
 * )
 */
class EventAndTicketingLinks extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $config = $this->configuration;
    $links = [];
    
    if (!empty($config['link_fields']['event'])) {
      $event_url = $row->getSourceProperty($config['link_fields']['event']);
      if (!empty($event_url)) {
        $links[] = [
          'title' => 'Event Url',
          'uri' => $event_url,
        ];
      }
    }
    if (!empty($config['link_fields']['ticketing'])) {
      $ticketing_url = $row->getSourceProperty($config['link_fields']['ticketing']);
      if (!empty($ticketing_url)) {
        $links[] = [
          'title' => 'Ticketing Url',
          'uri' => $ticketing_url,
        ];
      }
    }
    
    return $links;
  }
  
}

Additionally, since you ended up combining the urls into one source column anyway, you could use the hook or a plugin to work with that data too.

1

I eventually found THIS post on Drupal Stackexchange, and the answer proved to be extremely successful. I changed one of the columns in my source spreadsheet to follow the following format:

// Link Title|URL;Link Title|URL etc
// for example:
Event URL|https://coolvenue.com/etc;Ticketing URL|https://tickets.com/etc

You can see an example of the source data sheet here.

And then in my migration file, I used this:

...
    - name: src_links_delimited
      label: "LinksDelimited"
      selector: linksdelimited
...
  urls_temp:
    - plugin: skip_on_empty
      source: src_links_delimited
      method: process
      message: "No links. Skipping."
    - plugin: explode
      delimiter: ";"
    - plugin: explode
      delimiter: "|"
    - plugin: default_value
      default_value: ""
  field_links:
    - plugin: skip_on_empty
      method: process
      source: "@urls_temp"
      message: "No links. Skipping."
    - plugin: sub_process
      process:
        title: "0"
        uri: "1"
3
  • I know you said you didn't want to use custom process plugins, but you didn't say why. They're pretty straightforward to create and greatly improve the readability of your migration. It's cool that you figured this out, but I'll bet you spent more time than if you had created a process plugin and you're left with something far less readable. – sonfd Aug 6 '20 at 2:00
  • @sonfd yeah I definitely did spend more time than I would have liked looking for a straightforward yaml-only solution. I did create one process plugin for cleaning up filenames so their urls paths have no fugly escaped spaces in the url, but with regard to implementing one that takes two columns from the spreadsheet to create an array, I was fuzzy on where to start. Can you pass two variables to custom plugins or would I have to join them first, and split them in the plugin? I guess I was being lazy in not wanting to find out. But now I'm curious... – Doomd Aug 7 '20 at 23:43
  • 1
    Added this answer to address your comment. – sonfd Aug 8 '20 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.