2

We are trying to migrate content from an external data source into Drupal 8 using the core migrate modules. This is a stripped down example of the data being migrated:

id     resourceId     articleId     version     title          body
---------------------------------------------------------------------------
1      10             10            1           Page title     Page content
2      10             10            1.1         Page title     Page content
3      10             10            1.2         Page title     Page content
4      11             11            1           Page title     Page content
5      11             11            1.1         Page title     Page content

The id field is the unique identifier, and is the field set in the getIds() method, when as you can see the resourceId and articleId are more a preferred representation for a node id to be migrated into Drupal.

From my understanding, after reading through the migrate_example modules, the query() method must only return a single row; so when our migration runs, all these records are migrated as new nodes when in the previous database they were just revisions of one piece of content.

If we are intended to only return a single row in the query() method, what is the best way to only migrate the most recent version of each piece of content?

My first attempt was to try and achieve this in the processRow() method, but then if the migration is going through the table row by row, then how can you remove/update a previous row if the next row is a more recent version?

We have an option to just run our own script on the table first, to filter out all most recent versions and then run the migrate script on that, but I would like to know if there is a way to achieve this using the core modules in Drupal.

Thanks in advance.

2
  • You can check how node does it in d7_node_revision migration or you can compare the version of the existing node with the one in the row and if the row has newer verison return true or if it is older than the one in the db you can return false. If you want to store all revisions I guess the node migration holds the key to this.
    – user21641
    Nov 29, 2016 at 11:38
  • To expand a bit on Ivan's reference to the d7_node_revision migration, revisions are handled in a separate migration process with destination entity_revision:node. Please see that example configuration and the related d7_node_revision source plugin for more details.
    – Mike Ryan
    Nov 29, 2016 at 15:11

1 Answer 1

1

Yeah, this part is not documented at all, I had now also problems with it, so here is how I solved it:
When you want to migrate a content that has revisions, you need to migrate the first version with the entity:node plugin, then you migrate the next versions with the entity_revision:node. You don't do anything else, just change the destination plugin and when you migrate the revisions, you set the nid value to the first version of the node. So you keep the body and the file and other fields, you just add the nid field, so the revision plugin will know where to put the content. That's it at least from the yml part.
From the source, you have two one for the node plugin where you list only the first version of your content that you want to migrate and one for the revisions where you list everything EXCEPT the first version also, you have a new column which contains the first version's ID for every revision's it's own.

If you migrate the current publish one into the node table, and you migrate the rest into the revision, the IDs, the order of the revision and some other things will be messed up. At least my case was that.

By code (not 100% complette, just main parts):

migrate_plus.migration.node.yml:

id: my_node
source:
  plugin: my_node
process:
  title: title
  body: body
  langcode:
    plugin: default_value
    default_value: "und"
  status:
    plugin: default_value
    default_value: 1
  #other fields except nid.
destination:
  plugin: entity:node

Query for the source plugin (id = my_node):
SELECT id, title, body FROM content WHERE version = 1
in getIds() the id is the only id.

migrate_plus.migration.node_revision.yml:

id: my_node_revision
source:
  plugin: my_node_revision
process:
  nid:
    plugin: migration_lookup
    migration:
     - my_node
     - my_node_revision
    source: first_version
  title: title
  body: body
  langcode:
    plugin: default_value
    default_value: "und"
  status:
    plugin: default_value
    default_value: 1
  #other fields.
destination:
  plugin: entity_revision:node

Query for the source plugin (id = my_node_revision):

SELECT first_ver.id AS first_version, c.id, c.title, c.body FROM content c
  INNER JOIN (
    SELECT * FROM content c
    WHERE version = 1
  ) first_ver ON c.resourceId = first_ver.resourceId AND c.articleId = first_ver.articleId
WHERE c.version <> 1

in getIds() the id is the only id.

The above example will give new id's for the content.

EDIT: note that the example in this answer is a migration from confluence database into drupal, so therefore slightly different than a normal drupal-drupal migration. Also, if you look in the entity_revision:node plugin, which is located here: core/modules/migrate/src/Plugin/migrate/destination/EntityRevision.php you will notice that you have two ways of doing this.
A) Providing revision id.
B) Not providing revision id.

When you don't provide the revision ID, then as you add new revisions, it will always make the latest added revision the latest content version. (the example yml and source are like this above)

When you provide a revision ID, then you first need to add the current version into the node table, then add all the revisions and provide the revision ID, in that case, it won't make the last added revision as the latest content version.

5
  • If you also share the module, will be nice, seems pretty interesting
    – aniruddha
    Feb 18, 2021 at 7:27
  • Why entity is created out of first version? Shouldn't be created out of the highest version and other (lower) versions be used for revisions?
    – MilanG
    Apr 27, 2022 at 9:08
  • 1
    @MilanG you have to keep the chronological order, "If you migrate the current publish one into the node table" paragraphs tell it why. Apr 27, 2022 at 9:12
  • @golddragon007 ok, thanks. And how is specified which revision will be used as current one?
    – MilanG
    Apr 27, 2022 at 9:31
  • 1
    @MilanG The last added one by the entity_revision:node is adding. The query that is in the answer will return in historical order the record naturally, so the max revision ID will be added last which will be the actual latest version of the content. Also, I will edit my answer to make it more accurate. Apr 27, 2022 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.