4

I've written a bunch of migration script to migrate a site from Drupal 6 to Drupal 8.

On the development server I just ran a sample of each content type migration to test everything was OK while new content was still being created on production server on a daily basis by redactors.

During the validation time by the project team (which could take several weeks), I'll run migration everyday on a fresh database dump from production server. During that time also, the project team will need to create specific new content (content that is not being migrated) for different purposes, like testing brand new content type or creating specific base pages needed by the new site templates.

Since the migration process keeps the original ID, I need to find a way for new content to be created without taking the node ID from a node created on the production server. I’ve not been able to find any information about that case (which is quite a common case, I think) in the Drupal migration literature.

My first idea would be to change auto_increment value in relevant tables adding (let’s say) 100.000 to the current value, that way there wouldn’t be any collision between newly created content and the ones created the day before on production server. A brief look to the Drupal 8 database schema gave me the following tables and fields to alter:

  • file_managed => fid
  • node => nid
  • node_revision => vid
  • url_alias => pid

I am not sure the file managed fid field is kept during migration process; the same is true for the URL alias pid field since the link is made on the source field.

Do you think it would be enough to alter only the node and node_revision tables?

I also noticed a sequences table with a single value field with only one row. As far as I've read about this table, I've not really been able to figure out what it is used for. Maybe some of you can explain me and tell me if it's important regarding my issue.

Do you this is a correct approach? Are there other tables that could need to be altered?

  • 3
    Drupal doesn't require you to keep ID's, that's just the default configs and it is being discussed if that is good default or not. Combined with the migrate map and highwater mark, it is possible to migrate to new ID's and still be able to incrementally import as well as update new content on the source site. – Berdir Apr 25 '17 at 18:15
  • Thanks for your comment @Berdir.I wasn't aware of this. Can you please explain me how to change this default behaviour ? The only information I've been able to find is in this issue : drupal.org/node/2748609, but nothting really concrete for the moment.... – Kap Apr 25 '17 at 22:28
  • 1
    Not sure what your setup exactly is, I always use manually built migrate config entity with migrate_tools/migrate_plus, then you simply have to remove the "nid: nid" and "vid: vid" mapping (for node, similar for all other entities) and migration will just use automatically generated IDs – Berdir Apr 26 '17 at 18:31
  • Ouch ! So simple I couldn't even have been able to think to it... I'll test it... And if ever I need to update, everything will be kept in track with migrate map, is that correct ? – Kap Apr 26 '17 at 18:49
  • 1
    the mapping is there yet, but you need to specifiy a highwater field so that migrate can see if something changed. See drupal.org/node/2795403 – Berdir Apr 26 '17 at 22:09
1

Migrate, as shipped with Drupal 8, has two components:

  1. The migrate module. This provides a very flexible API.
  2. The migrate_drupal module. This provides Drupal 6 / 7 to 8 migrations (the migrations themselves has been scattered into the modules). This is the part which keeps IDs and the motivation for this was very simple: there's just no way to tell where those IDs might lurk. Say, one of the old nodes has a http://example.com/node/12345 link in the body text and we changed the node id to 67890 during migration? Then that link breaks. It also made for more rapid development which is not a bad side effect either. It is arguable whether for example file IDs deserved such treatment or not.

As @berdir comments, if you remove the nid: nid process definition from a node migration then it will happily generate new node IDs for you, no sweat but you need to look everywhere for the same to make sure that any references are changed to a migration_lookup process definition. Some of it is already done for you, some of it is not.

For example, comment/migration_templates/d6_comment.yml has:

# If you are using this file to build a custom migration consider removing
# the cid field to allow incremental migrations.
cid: cid
pid:
  plugin: migration_lookup
  migration: d6_comment
  source: pid

For core, pid: pid would work just as well but instead, this migration is already prepared for this migration creating new IDs. But then it continues with

entity_id: nid

Now the corresponding node/migration_templates/d6_node.yml has

# If you are using this file to build a custom migration consider removing
# the nid and vid fields to allow incremental migrations.
nid: tnid
vid: vid

If you actually do that then you need to circle back to comment/migration_templates/d6_comment.yml and write

entity_id:
  plugin: migration_lookup
  migration: 
    - d6_node:article
    - d6_node:page
  source: nid

Because the actual migration is not d6_node but rather d6_node:nodetype.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.