Webform Error Occurs During Configuration Import

Question

Currently we are having some trouble handling webform module configuration on our various environments.

When running a configuration import drupal config:import --directory=$SYNC_DIR on our dev environments with a fresh database from production we receive the following error:

[ERROR] An error occurred while trying to write the config file: "Entities exist of type Webform submission and Webform Borough Newsletter. These entities need to be deleted before importing."

This particular web form (Borough Newsletter) has not been exported as config from production and has no associated yaml file, additionally it has some submissions in the database. When we remove all submissions of this particular webform the import completes, but removes any webforms not stored as config (no surprise there). We also tried to use the Config Ignore to prevent webforms from importing configuration, but we still got the above error. I have three main questions:

1. Our webforms are created on production by non-developers. Is the proper workflow to start exporting webform config from prod and bring it to our dev environments top down?

2. What exactly was the cause of the error we got during import (mentioned above)?

3. How come we still get an import error when we explicitly attempted to block any webform config from being imported? Was it due to a dependency on webforms from some other configuration item?

Webform Module Version: 8.x-5.3

Drupal Core Version: 8.6.17

Thank you in advance!

Answer 1

Ad 1)
I guess the proper workflow is never do any config changes on prod, completely disable all config UIs on prod and do only one-way forward-syncs from stage to prod. But I'm also working for small businesses, who do their work directly on the prod server, so I also can't use the "proper" workflow.

My personal workflow is

1. Prod always is the source of truth (no config ignore).
2. Before development, always pull config from prod.
   drush @prod site:ssh drush_cex_and_git_push.sh "Backsync"
git_pull_and_drush_cim.sh
3. During development, don't change config on prod to avoid merging troubles.
4. After development, push to repository and pull changes on prod.
   drush_cex_and_git_push.sh "New awesome feature"
drush @prod site:ssh git_pull_and_drush_cim.sh

This workflow is NOT safe for concurrent config changes, so this will not work well if you have frequent config changes on prod or long-term or multi-person development jobs!

Prerequisites:

• SSH access on prod. Add dev machine's SSH key on prod's ~/.ssh/authorized_keys.
• Create drush site environment entries for @dev and @prod in ~/drush/sites/self.site.yml to execute drush commands on prod environment while working on dev machine. (doc)
• Use git with SSH keys for quick syncing config between environments without password prompts.
• Use some bash scripts for single-command syncs between environments

git_pull_and_drush_cim.sh

#!/usr/bin/env sh
echo ""
echo "PULL and IMPORT all updates..."
echo ""

git pull
echo ""
echo "DELETING unused config files"
echo ""
git clean -fd config/sync
echo ""
./composer.phar install --no-dev
./vendor/bin/drush state:set system.maintenance_mode 1 -y
./vendor/bin/drush cache:rebuild
./vendor/bin/drush updatedb -y
./vendor/bin/drush config:import -y
./vendor/bin/drush core:cron -y
./vendor/bin/drush state:set system.maintenance_mode 0 -y
./vendor/bin/drush cache:rebuild

drush_cex_and_git_push.sh

#!/usr/bin/env sh

if [ -z "$1" ]; then
  echo "Usage: drush_cex_and_git_push.sh <commit message>"
  echo ""
  exit 1
fi


echo ""
echo "EXPORT and PUSH all updates..."
echo "Commit Message: $1"
echo ""

./vendor/bin/drush config:export -y
git add -A
git commit -m "$1"
git push

I'm also using those scripts as part of my semi-automated maintenaince script running composer updates for all my projects.

---

Ad 2)
drush config:import deletes all existing configs from the DB that are not present in the sync directory. Since you cannot delete an entity config while there are dependent entity instances in the database, you get this error message.
It would be the same as if deleting node.type.article.yml config while still having article nodes in the database.

---

There are a lot of blog posts on large enterprise projects with a strict dev --> stage --> prod workflow. I'd love to see more info on how to handle a small dev <--> prod workflow.

Answer 2

UPDATE 2: I posted a question on Drush's GitHub concering this issue and have received an answer that clears it up.

Essentially, Config Ignore only modifies the default sync storage and when using config-import with no args it takes effect, passing the source argument changes the default sync storage and this is not affected by config-import. More details can be found here.

---

UPDATE 1: It turns out that the module Webform Config Ignore (and Config Ignore) depends on, Config Filter, doesn't yet support Drupal Console so that solves the mystery of why that command didn't respect the rules for ignoring webform configuration.

Still looking into why drush config-import --source differs from drush config-import

---

So after some investigation and a little luck we discovered the issue, though why it occurred is something we're still trying to figure out.

In our deploy process we run a db update (drush -y updb --entity-updates) and then run drupal config:import --directory=$SYNC_DIR to start the configuration import. As I mentioned earlier even with the Config Ignore (or Webform Config Ignore) module the webform config still attempts to import and the error occurs. An external vendor we are working with on this site had a similar deploy script, but they used the command drush -y config-import to run the config import process, when that command is used the webform config is actually ignored and the process completes.

It gets even stranger, if the command drush -y config-import --source=$SYNC_DIR is used it fails with the same error as the drupal console command. Only when using the drush config-import command with no args (so it uses the sync dir within settings.php, which is the same as $SYNC_DIR) does the configuration import actually ignore webform configuration and complete successfully.

I'll post an update when I figure out why is this is the case.

Drush Version: 8.3.0

Drupal Console Version: 1.9.1