How to refresh new migrations in Drupal 8 migration module?

Question

When writing Drupal 8 migrate processes I only can refresh new migration processes if I reinstall the custom migration module. drush cr doesn't work.

Explanation:

I have one module, ie: cm_migrations, which have 2 migrations:

• cm_users
• cm_tags

With drush ms I see:

Group: beer     Status  Total  Imported  Unprocessed  Last imported       
 cm_users        Idle    3      3         0            2016-02-09 16:56:25 
 cm_tags         Idle    4      0         4                                

I write a new migrate process:

• cm_beers

I execute drush cr; drush ms and I see the same table as previous.

Only when I uninstall and install cm_migrations I can see the three migration processes:

Group: beer     Status  Total  Imported  Unprocessed  Last imported       
 cm_users        Idle    3      3         0            2016-02-09 16:56:25 
 cm_tags         Idle    4      0         4                          
 cm_beers        Idle    4      0         4              

Any clue about how can I refresh the migration processes without reinstall the module?

Thanks

Answer 1

I found this worked for me

drush config-import --partial --source=modules/custom/migrate_module/config/install/

Answer 2

Migrations should be in the MODULENAME/migrations subdirectory. After modifying a migration config a simple drush cr suffices to apply the changes.

A config deriver will turn a migration config back into a migration plugin.

See this change record: Migrations are plugins instead of configuration entities

Answer 3

There is a module for that. Since normally the yml files are only loaded upon install.

This module lets you can define what module config files are automaticly imported.

https://www.drupal.org/project/config_devel

Also see this blog post: https://blog.liip.ch/archive/2016/05/04/using-the-new-drupal-8-migration-api-module.html

Developing your own Drupal 8 migrate modules and fighting caching issues

You have learned, that the whole migration mapping is now done in yaml files. But how about writing your own migration yaml files?

Unfortunately, there are some pitfalls for new Drupal 8 developers. Because of the Configuration Management Interface (https://www.drupal.org/documentation/administer/config) of Drupal 8, all yml files in the “config/install” directory are only imported when installing the module.

This is very impractical if you want to develop new configuration files. To address this, a module “Configuration Development” (https://www.drupal.org/project/config_devel) which resolves the caching issues can be installed. It is possible to import certain yml files on every request. But unfortunately drush commands are not supported yet. So we need to add all yaml files we want to import into a new section in our module.info.yml.

config_devel:
 install:
   - migrate_plus.migration.page_node
   - migrate_plus.migration.menu_item
   - migrate_plus.migration_group.liip

Then we can run the following commands after updating the yml file. This will import the new configuration file into Configuration Management Interface.

drush cdi <module_name>
drush cr

Answer 4

I was struggling with the same issue: how to update migration config during development. Here is what I ended up with:

drush cdi1 path-to-config-dir-form-site-root/your-config.yml

config-devel-import-one, command (cdi1 for short) is intended to import single config into active configuration. It is the part of config_devel module mentioned above.

Answer 5

I didn't have success with FLY's solution. Had to uninstall migrate_tools also, to get yaml's refreshed. And no luck with devel_config.

I did find this refreshed configurations:

$ drush @site_alias pmu mymodule, migrate_tools, migrate_plus -y
$ drush @site_alias en mymodule, migrate_tools, migrate_plus -y

Answer 6

Following up on Peter J Lord's solution, you can also add a config directory (for new or updated ymls) to your settings.php file. e.g.:

$config_directories['fresh'] = 'sites/default/fresh';

Place new yml files in that directory and then import the fresh configuration settings with drush's config-import command:

drush cim fresh --partial 

Answer 7

I have found that sometimes none of the above work. Even combining them all. However I could trick migrate_tools into giving me some insight.

Apparently there is a check if the yml file is valid before it gets listed. Try running the importer (yes, even if it doesn't show up with $ drush ms )

My migration was called products, so I did: $ drush mim products

Migration failed with source plugin exception: The "int" plugin does not exist.

Fixing this fault (changing int to integer in my case) and then following @FLY 's answer above made the migration show up. Shame the error did not show up and it just failed silently, but that's another topic. ;-)

P.S. I would have posted this as a comment, but I don't have enough points yet.

Answer 8

In the answer by FLY above, drush cdi <module_name> should now be drush cde <module_name>.