1

We are trying to migrate content from an external data source into Drupal 8 using the core migrate modules. This is a stripped down example of the data being migrated:

id     resourceId     articleId     version     title          body
---------------------------------------------------------------------------
1      10             10            1           Page title     Page content
2      10             10            1.1         Page title     Page content
3      10             10            1.2         Page title     Page content
4      11             11            1           Page title     Page content
5      11             11            1.1         Page title     Page content

The id field is the unique identifier, and is the field set in the getIds() method, when as you can see the resourceId and articleId are more a preferred representation for a node id to be migrated into Drupal.

From my understanding, after reading through the migrate_example modules, the query() method must only return a single row; so when our migration runs, all these records are migrated as new nodes when in the previous database they were just revisions of one piece of content.

If we are intended to only return a single row in the query() method, what is the best way to only migrate the most recent version of each piece of content?

My first attempt was to try and achieve this in the processRow() method, but then if the migration is going through the table row by row, then how can you remove/update a previous row if the next row is a more recent version?

We have an option to just run our own script on the table first, to filter out all most recent versions and then run the migrate script on that, but I would like to know if there is a way to achieve this using the core modules in Drupal.

Thanks in advance.

  • You can check how node does it in d7_node_revision migration or you can compare the version of the existing node with the one in the row and if the row has newer verison return true or if it is older than the one in the db you can return false. If you want to store all revisions I guess the node migration holds the key to this. – user21641 Nov 29 '16 at 11:38
  • To expand a bit on Ivan's reference to the d7_node_revision migration, revisions are handled in a separate migration process with destination entity_revision:node. Please see that example configuration and the related d7_node_revision source plugin for more details. – Mike Ryan Nov 29 '16 at 15:11
0

Yeah, this part is not documented at all, I had now also problems with it, so here is how I solved it:
When you want to migrate a content that has revisions, you need to migrate the first version with the entity:node plugin, then you migrate the next versions with the entity_revision:node. You don't do anything else, just change the destination plugin and when you migrate the revisions, you set the nid value to the first version of the node. So you keep the body and the file and other fields, you just add the nid field, so the revision plugin will know where to put the content. That's it at least from the yml part.
From the source, you have two one for the node plugin where you list only the first version of your content that you want to migrate and one for the revisions where you list everything EXCEPT the first version also, you have a new column which contains the first version's ID for every revision's it's own.

If you migrate the current publish one into the node table, and you migrate the rest into the revision, the IDs, the order of the revision and some other things will be messed up. At least my case was that.

By code (not 100% complette, just main parts):

migrate_plus.migration.node.yml:

id: my_node
source:
  plugin: my_node
process:
  title: title
  body: body
  langcode:
    plugin: default_value
    default_value: "und"
  status:
    plugin: default_value
    default_value: 1
  #other fields except nid.
destination:
  plugin: entity:node

Query for the source plugin (id = my_node):
SELECT id, title, body FROM content WHERE version = 1
in getIds() the id is the only id.

migrate_plus.migration.node_revision.yml:

id: my_node_revision
source:
  plugin: my_node_revision
process:
  nid:
    plugin: migration_lookup
    migration:
     - my_node
     - my_node_revision
    source: first_version
  title: title
  body: body
  langcode:
    plugin: default_value
    default_value: "und"
  status:
    plugin: default_value
    default_value: 1
  #other fields.
destination:
  plugin: entity_revision:node

Query for the source plugin (id = my_node_revision):

SELECT first_ver.id AS first_version, c.id, c.title, c.body FROM content c
  INNER JOIN (
    SELECT * FROM content c
    WHERE version = 1
  ) first_ver ON c.resourceId = first_ver.resourceId AND c.articleId = first_ver.articleId
WHERE c.version <> 1

in getIds() the id is the only id.

The above example will give new id's for the content.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.