We are trying to migrate content from an external data source into Drupal 8 using the core migrate modules. This is a stripped down example of the data being migrated:
id resourceId articleId version title body
---------------------------------------------------------------------------
1 10 10 1 Page title Page content
2 10 10 1.1 Page title Page content
3 10 10 1.2 Page title Page content
4 11 11 1 Page title Page content
5 11 11 1.1 Page title Page content
The id field is the unique identifier, and is the field set in the getIds() method, when as you can see the resourceId and articleId are more a preferred representation for a node id to be migrated into Drupal.
From my understanding, after reading through the migrate_example modules, the query() method must only return a single row; so when our migration runs, all these records are migrated as new nodes when in the previous database they were just revisions of one piece of content.
If we are intended to only return a single row in the query() method, what is the best way to only migrate the most recent version of each piece of content?
My first attempt was to try and achieve this in the processRow() method, but then if the migration is going through the table row by row, then how can you remove/update a previous row if the next row is a more recent version?
We have an option to just run our own script on the table first, to filter out all most recent versions and then run the migrate script on that, but I would like to know if there is a way to achieve this using the core modules in Drupal.
Thanks in advance.