How do I migrate file entities into media entities?

Question

I am using the Migrate module for a D7 to D8 migration and I am writing the whole migration manually in code (rather than use the built in D7 migrate module, as I wanted more granual control on the migration.)

I have the following structure: the D7 site has a image field where images are stored as File entities. On the D8 site, the image field is an entity reference to a Media entity (and the Media entity in turn has a Image field.)

Originally, I had the following for my Images migration:

id: image_files

source:
  plugin: legacy_images
  constants:
    source_base_path: http://example.com/

destination:
  plugin: 'entity:file'

process:
  fid: fid
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
    -     constants/source_base_path
    -     uri
    -
      plugin: urlencode
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - uri
  filemime: filemime
  status: status

Inside my article node migration file, I had the following:

'field_article_image/target_id':
plugin: migration
migration: image_files
source: field_article_image 

but I realized that this would not work. The target_id coming from the image_files migration were really File Entity ID's, not media entity ID's. In the ideal world, I'd like to find a way to create a third migration, which would create this middle step, and migrate file entities into Media Entities and then map that migration to the Articles migration. However, I can't figure out a good way of doing this.

Plan B will simply be to create a process plugin for the images migration, which will manually create file entities, attach them to media entities, and pass that migration over to Articles (this removes the middle step). This would mean, though, that while Media entities can be rolled back, File Entities could not.

Answer 1

I ended up chosing to do it slightly different - I create a regular file import, set that migration as the source for my media entity reference field, and then applied a second process plugin 'MediaGenerate' to translate the FID to the new media target_id

<?php

namespace Drupal\my_migration\Plugin\migrate\process;

use Drupal\media_entity\Entity\Media;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\MigrateException;
use Drupal\migrate\Row;
use Drupal\migrate_plus\Plugin\migrate\process\EntityGenerate;

/**
 * Generate a media entity with specified metadata.
 *
 * This plugin is to be used by migrations which have media entity reference
 * fields.
 *
 * Available configuration keys:
 * - destinationField: the name of the file field on the media entity.
 *
 * @code
 * process:
 *   'field_files/target_id':
 *     -
 *       plugin: migration
 *       source: files
 *     -
 *       plugin: media_generate
 *       destinationField: image
 *
 * @endcode
 *
 * @MigrateProcessPlugin(
 *   id = "media_generate"
 * )
 */
class MediaGenerate extends EntityGenerate {

/**
* {@inheritdoc}
*/
public function transform($value, MigrateExecutableInterface $migrateExecutable, Row $row, $destinationProperty) {
if (!isset($this->configuration['destinationField'])) {
  throw new MigrateException('Destination field must be set.');
}
// First load the target_id of the file referenced via the migration.
/* @var /Drupal/file/entity/File $file */
$file = $this->entityManager->getStorage('file')->load($value);

if (empty($file)) {
  throw new MigrateException('Referenced file does not exist');
}

// Creates a media entity if the lookup determines it doesn't exist.
$fileName = $file->label();
if (!($entityId = parent::transform($fileName, $migrateExecutable, $row, $destinationProperty))) {
  return NULL;
}
$entity = Media::load($entityId);

$fileId = $file->id();
$entity->{$this->configuration['destinationField']}->setValue($fileId);
$entity->save();

return $entityId;
}

}

Answer 2

As Media is an entity type you should create its own migration. You can generate a new source from the file table. Here is an example

https://gist.github.com/jibran/8e7cd2319e873858dd49a272227a4fd2

Then with migration_lookup you can map the fields like this.

field_d8_media_image/0/target_id:
  plugin: migration_lookup
  migration: my_media_image
  source: field_d7_image/0/fid

Answer 3

I appreciated the accepted answer a lot, however it already had some deprecated definitions and did not support posting alt and title image properties. Thus, I enhanced it a bit to support that and to work smoothly with latest Drupal 8.6.x. Here is the code of MediaGenerate.php (the appropriate Yaml syntax is inside the doc comment):

<?php

namespace Drupal\my_migration\Plugin\migrate\process;

use Drupal\media\Entity\Media;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\MigrateException;
use Drupal\migrate\Row;
use Drupal\migrate_plus\Plugin\migrate\process\EntityGenerate;

/**
 * Generate a media entity with specified metadata.
 *
 * This plugin is to be used by migrations which have media entity reference
 * fields.
 *
 * Available configuration keys:
 * - destinationField: the name of the file field on the media entity.
 *
 * @code
 * process:
 *   'field_files/target_id':
 *     -
 *       plugin: migration_lookup
 *       migration: my_file_migration
 *       source: field_image/0/fid
 *     -
 *       plugin: media_generate
 *       destinationField: image
 *       imageAltSource: field_image/0/alt
 *       imageTitleSource: field_image/0/title
 *
 * @endcode
 *
 * If image_alt_source and/or image_title_source configuration parameters
 * are provided, alt and/or title image properties will be fetched from provided
 * source fields (if available) and pushed into media entity
 *
 * @MigrateProcessPlugin(
 *   id = "media_generate"
 * )
 */
class MediaGenerate extends EntityGenerate {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrateExecutable, Row $row, $destinationProperty) {
    if (!isset($this->configuration['destinationField'])) {
      throw new MigrateException('Destination field must be set.');
    }

    // First load the target_id of the file referenced via the migration.
    /* @var /Drupal/file/entity/File $file */
    $file = $this->entityManager->getStorage('file')->load($value);

    if (empty($file)) {
      throw new MigrateException('Referenced file does not exist');
    }

    // Creates a media entity if the lookup determines it doesn't exist.
    $fileName = $file->label();
    if (!($entityId = parent::transform($fileName, $migrateExecutable, $row, $destinationProperty))) {
      return NULL;
    }

    $entity = Media::load($entityId);

    $fileId = $file->id();

    $destinationFieldValues = $entity->{$this->configuration['destinationField']}->getValue();
    $destinationFieldValues[0]['target_id'] = $fileId;

    $this->insertPropertyIntoDestinationField($destinationFieldValues, $row, 'alt', 'imageAltSource');
    $this->insertPropertyIntoDestinationField($destinationFieldValues, $row, 'title', 'imageTitleSource');

    $entity->{$this->configuration['destinationField']}->setValue($destinationFieldValues);
    $entity->save();

    return $entityId;
  }

  protected function insertPropertyIntoDestinationField(array &$destinationFieldValues, Row $row, $propertyKey, $configurationKey) {
    // Set alt and title into media entity if not empty
    if (isset($this->configuration[$configurationKey])) {
      $propertyValue = $row->getSourceProperty($this->configuration[$configurationKey]);
      if (!empty($propertyValue)) {
        $destinationFieldValues[0][$propertyKey] = $propertyValue;
      }
    }
  }
}

Answer 4

If you want to migrate files in Drupal 8 to media entities you can use this module: https://www.drupal.org/project/migrate_file_to_media

It has a drush script, that automatically creates the media reference fields. Additionally it detects duplicate images using a binary hash. And it supports translations.