1

EDIT: To Answer my own question...

I ended up making multiple migration files like this:

  1. Node Type A Part 1 (most fields migrated except the ones that need to reference Node Type B)
  2. Node Type B
  3. Node Type A Part 2 (now you can add the field referencing Node Type B)
  4. etc.

Here's a Github link to my finalized set of real-world migration files which illustrate how I broke up my migration into different sequential parts. Good luck!


Original Question:

My Drupal migration yaml files are not being imported in the correct execution order.

I have 8 migration files, most that depend on another file being executed first. I'm using a Google Spreadsheet as the source, and targeting Drupal 8.9+. I have tagged all my dependencies, and all was going well with my test imports (using drush migrate:import --tag="Rooster Business") and the corresponding rollback commands. But at some point, after adding a few fields, the migrations began to fail because they were being executed out of order (particularly my business_locations and business_offers migrations).

Here's the exact error:

[error]  Migration rooster_node_business_offers did not meet the requirements. 
Missing migrations rooster_node_business_location. requirements: rooster_node_business_location.

The new fields themselves don't fail, so it's not that (I wouldn't think). I think I have correctly set up the dependencies in all the files, but they are NOT being respected (at least not completely).

I have tried the following:

  • Alphabetizing all the migration id's and filenames (to order them by desired import execution order).
  • Dropping all migration tables from the database and doing full cache refresh
  • Changing the last file I want executed (rooster_node_business_offers) to use "optional" dependencies instead of "required." (This causes the failure to go away, but then some of the desired field values are missing from the migration).
  • Uninstalling and reinstalling the entire custom migration module.
  • I've tried just migrating the "last" migration with drush migrate:import rooster_node_business_offers --execute-dependencies command which should in theory execute all the files in order as they all depend in a chain on each other. No dice. I get the same error: [error] Migration rooster_node_business_offers did not meet the requirements...

None of these attempts have solved the problem. I have a theory that it has something to do with the fields names of the sources (because I think I may have changed a few of them, and have definitely added quite a few). But that still doesn't account for why the migrations are not respecting the dependency order.

Bottom line: I have two questions:

  1. Why are the last two migration files (rooster_node_business_location & rooster_node_business_offers) not being executed in order by dependency (and how can I fix this)?
  2. If that is not an easy to answer question, what is the logic of the "default" ordering, and how can I game that logic so I can still get my migration files to execute in the desired order?

Here is the order I'd LIKE to have the migrations to execute in:

rooster_file_bg_photo
rooster_file_logo
rooster_users
rooster_file_logo_to_media
rooster_file_bg_photo_to_media
rooster_node_parent_business
rooster_node_business_location
rooster_node_business_offers

And here is the order that they are actually being migrated in (same as the order displayed after entering the drush ms command:

  Rooster (Rooster)   rooster_file_bg_photo            Idle
  Rooster (Rooster)   rooster_file_logo                Idle
  Rooster (Rooster)   rooster_users                    Idle
  Rooster (Rooster)   rooster_file_logo_to_media       Idle
  Rooster (Rooster)   rooster_file_bg_photo_to_media   Idle
  Rooster (Rooster)   rooster_node_parent_business     Idle
  Rooster (Rooster)   rooster_node_business_offers     Idle
  Rooster (Rooster)   rooster_node_business_location   Idle

I've made a repository of my exact migration files (and I've only redacted the Google Spreadsheet URLs). Here are all the migration files on Github.

Thanks in advance for your help!

10
  • A migration can't be optional if you use its ID in this migration with migration_lookup. And you have a circular dependency between your last two migrations.
    – Hudri
    Aug 14 '20 at 7:36
  • 1
    Simply specify nid (or uid key in case of user entity) in the process section of migration #1B, e.g. through a migration_lookup or entity_lookup, then you can update an existing entity.
    – Hudri
    Aug 14 '20 at 16:37
  • 1
    So, I did some more research. As I suspected, you ARE allowed to reference migration_lookups for migrations that are imported afterwards. BUT, there is a bug. The id of the migration file needed in subsequent references has to be in REVERSE alphabetical order: drupal.org/project/drupal/issues/3024634
    – Doomd
    Aug 14 '20 at 20:48
  • 1
    @miststudent2011 I will have to go back and look at the exact migration files, but I recall that I just made another file that basically filled in the missing lookups after the dependancy was filled in. So...if part of A needs to lookup something populated by B, make a C file that runs after B is complete. I'll share my files after I get into work. Cheers.
    – Doomd
    Jan 19 '21 at 11:36
  • 1
    @miststudent2011FYI, I updated my question with a link to a github repo I just created with all my finalized migration files. You'll notice that I broke up the migration into two parts for two different node types because I wanted them to reference each other. Hope that helps
    – Doomd
    Jan 21 '21 at 2:02
0

I was able to solve the migration dependency unmet error.

Here is my approach to resolve the issue.

For example you have three migrations migration_a , migration_b and migration_c , and migration_a is dependent on migration_b and migration_b is dependent on migration_c.

What you have to do is define migration_b as optional dependency in migration_a and similarly define migration_c as optional dependency in migration_a .

Then import the migration_c manually first with the command drush mim migration_c and later migration_b with command drush mim migration_b and finally drush mim migration_a

Thus you have your migration imported.

Please let me know if you have any additional suggestions to resolve the issue.


There are other solutions which are scattered across the drupal.org to resolve the issue but they did not help me to resolve the issue.

  1. Name the migrations in the alphabetical order respective to their migration order.
  2. Check whether the line is defined $settings['cache']['default'] = 'cache.backend.null'; relevant bug

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.