How can I ensure migration YAML files execute in specified dependency order?

Question

EDIT: To Answer my own question...

I ended up making multiple migration files like this:

1. Node Type A Part 1 (most fields migrated except the ones that need to reference Node Type B)
2. Node Type B
3. Node Type A Part 2 (now you can add the field referencing Node Type B)
4. etc.

Here's a Github link to my finalized set of real-world migration files which illustrate how I broke up my migration into different sequential parts. Good luck!

---

Original Question:

My Drupal migration yaml files are not being imported in the correct execution order.

I have 8 migration files, most that depend on another file being executed first. I'm using a Google Spreadsheet as the source, and targeting Drupal 8.9+. I have tagged all my dependencies, and all was going well with my test imports (using drush migrate:import --tag="Rooster Business") and the corresponding rollback commands. But at some point, after adding a few fields, the migrations began to fail because they were being executed out of order (particularly my business_locations and business_offers migrations).

Here's the exact error:

[error]  Migration rooster_node_business_offers did not meet the requirements. 
Missing migrations rooster_node_business_location. requirements: rooster_node_business_location.

The new fields themselves don't fail, so it's not that (I wouldn't think). I think I have correctly set up the dependencies in all the files, but they are NOT being respected (at least not completely).

I have tried the following:

• Alphabetizing all the migration id's and filenames (to order them by desired import execution order).
• Dropping all migration tables from the database and doing full cache refresh
• Changing the last file I want executed (rooster_node_business_offers) to use "optional" dependencies instead of "required." (This causes the failure to go away, but then some of the desired field values are missing from the migration).
• Uninstalling and reinstalling the entire custom migration module.
• I've tried just migrating the "last" migration with drush migrate:import rooster_node_business_offers --execute-dependencies command which should in theory execute all the files in order as they all depend in a chain on each other. No dice. I get the same error: [error] Migration rooster_node_business_offers did not meet the requirements...

None of these attempts have solved the problem. I have a theory that it has something to do with the fields names of the sources (because I think I may have changed a few of them, and have definitely added quite a few). But that still doesn't account for why the migrations are not respecting the dependency order.

Bottom line: I have two questions:

1. Why are the last two migration files (rooster_node_business_location & rooster_node_business_offers) not being executed in order by dependency (and how can I fix this)?
2. If that is not an easy to answer question, what is the logic of the "default" ordering, and how can I game that logic so I can still get my migration files to execute in the desired order?

Here is the order I'd LIKE to have the migrations to execute in:

rooster_file_bg_photo
rooster_file_logo
rooster_users
rooster_file_logo_to_media
rooster_file_bg_photo_to_media
rooster_node_parent_business
rooster_node_business_location
rooster_node_business_offers

And here is the order that they are actually being migrated in (same as the order displayed after entering the drush ms command:

  Rooster (Rooster)   rooster_file_bg_photo            Idle
  Rooster (Rooster)   rooster_file_logo                Idle
  Rooster (Rooster)   rooster_users                    Idle
  Rooster (Rooster)   rooster_file_logo_to_media       Idle
  Rooster (Rooster)   rooster_file_bg_photo_to_media   Idle
  Rooster (Rooster)   rooster_node_parent_business     Idle
  Rooster (Rooster)   rooster_node_business_offers     Idle
  Rooster (Rooster)   rooster_node_business_location   Idle

I've made a repository of my exact migration files (and I've only redacted the Google Spreadsheet URLs). Here are all the migration files on Github.

Thanks in advance for your help!

Answer 1

I was able to solve the migration dependency unmet error.

Here is my approach to resolve the issue.

For example you have three migrations migration_a , migration_b and migration_c , and migration_a is dependent on migration_b and migration_b is dependent on migration_c.

What you have to do is define migration_b as optional dependency in migration_a and similarly define migration_c as optional dependency in migration_a .

Then import the migration_c manually first with the command drush mim migration_c and later migration_b with command drush mim migration_b and finally drush mim migration_a

Thus you have your migration imported.

Please let me know if you have any additional suggestions to resolve the issue.

---

There are other solutions which are scattered across the drupal.org to resolve the issue but they did not help me to resolve the issue.

1. Name the migrations in the alphabetical order respective to their migration order.
2. Check whether the line is defined $settings['cache']['default'] = 'cache.backend.null'; relevant bug