Config split is imported only every second time of jenkins job run

Question

I have some weird behavior with drush cim when running from a jenkins job. It seems like a config split is only imported every second time or so when the job is run. The update skript containing the drush cim does work fine on local ddev sites and with other splits on other environements.

Jenkins does the following:

export stage=dev
sites=$(find "$project_dir"/web/sites/ -name "*.domain.de" -type l -printf "%f\n")
update_script="$project_dir/custom_scripts/update.sh"
parallel --jobs 4 --env stage --tag --verbose --line-buffer --keep-order ${update_script} ::: $sites

update.sh does (shortened) the following:

...
drush -l ${site} updatedb --no-cache-clear -y
drush -l ${site} cr -y
drush -l ${site} cim -y

#If needed do some language imports
#drush -l ${site} locale:check -y
#drush -l ${site} locale:update -y
#drush -l ${site} locale:import de ../translations/custom-translations.de.po --type=customized --override=all -y

drush -l ${site} cr -y
drush -l ${site} cim -y
drush -l ${site} deploy:hook -y
drush -l ${site} cr -y

In settings.php there is a if statement that checks for stage=dev

if (preg_match('/local.intern.domain.de|.ddev.local|.ddev.site$/im', getenv('VIRTUAL_HOST'),$matches) ||
 preg_match('/dev$/im', getenv('stage'),$matches)) {
  if (getenv('stage')) {
    print_r('Inside IF Current stage: ' . getenv('stage'));
  }
  $config['config_split.config_split.config_dev']['status'] = TRUE;
...

While debugging this I can see the if statement is correctly entered and I get the following in my Jenkins job log, in this case the config_split.config_dev ist correctly applied:

sub.domain.de    [debug] Try to validate bootstrap phase 5 [1.17 sec, 24.04 MB]
sub.domain.de    [debug] Calling Drush\Drupal\Commands\config\ConfigImportCommands::doImport(Drupal\Core\Config\StorageComparer) [5.69 sec, 116.35 MB]
sub.domain.de   Current stage: devInside IF Current stage: devCurrent stage: devInside IF Current stage: dev+-------------+----------------------------------+-----------+
sub.domain.de   | Collection  | Config                           | Operation |
sub.domain.de   +-------------+----------------------------------+-----------+
sub.domain.de   |             | dblog.settings                   | Create    |
sub.domain.de   |             | devel.settings                   | Create    |
sub.domain.de   |             | devel.toolbar.settings           | Create    |
sub.domain.de   |             | system.menu.devel                | Create    |
sub.domain.de   |             | ultimate_cron.job.dblog_cron     | Create    |
sub.domain.de   |             | views.view.watchdog              | Create    |
sub.domain.de   |             | coffee.configuration             | Create    |
sub.domain.de   |             | stage_file_proxy.settings        | Create    |
sub.domain.de   |             | core.extension                   | Update    |
sub.domain.de   |             | entity_print.print_engine.dompdf | Update    |
sub.domain.de   | language.en | system.menu.devel                | Create    |
sub.domain.de   | language.en | views.view.watchdog              | Create    |
sub.domain.de   +-------------+----------------------------------+-----------+
sub.domain.de   
sub.domain.de    // Import the listed configuration changes?: yes.                              
sub.domain.de   
sub.domain.de    [debug] config_pages commands loaded even though its constraint (^9) is incompatible with Drush 10.6.1. Broaden the constraint in modules/contrib/config_pages/composer.json (see 'extra\drush\services' section) to remove this message. [6.94 sec, 135.3 MB]
sub.domain.de    [debug] Found drush.services.yml for config_split Drush commands [6.94 sec, 135.3 MB]

The next time I get this:

sub.domain.de    // Do you wish to run the specified pending updates?: yes.                     
sub.domain.de   
sub.domain.de   Current stage: devInside IF Current stage: devdev
sub.domain.de   Current stage: devInside IF Current stage: devCurrent stage: devInside IF Current stage: devCurrent stage: devInside IF Current stage: devdev
sub.domain.de   Current stage: devInside IF Current stage: dev+-------------+----------------------------------+-----------+
sub.domain.de   | Collection  | Config                           | Operation |
sub.domain.de   +-------------+----------------------------------+-----------+
sub.domain.de   |             | core.extension                   | Update    |
sub.domain.de   |             | entity_print.print_engine.dompdf | Update    |
sub.domain.de   |             | stage_file_proxy.settings        | Delete    |
sub.domain.de   |             | coffee.configuration             | Delete    |
sub.domain.de   |             | views.view.watchdog              | Delete    |
sub.domain.de   |             | ultimate_cron.job.dblog_cron     | Delete    |
sub.domain.de   |             | system.menu.devel                | Delete    |
sub.domain.de   |             | devel.toolbar.settings           | Delete    |
sub.domain.de   |             | devel.settings                   | Delete    |
sub.domain.de   |             | dblog.settings                   | Delete    |
sub.domain.de   | language.en | views.view.watchdog              | Delete    |
sub.domain.de   | language.en | system.menu.devel                | Delete    |
sub.domain.de   +-------------+----------------------------------+-----------+
sub.domain.de   
sub.domain.de    // Import the listed configuration changes?: yes.                              
sub.domain.de   
sub.domain.de    [success] No pending deploy hooks.
sub.domain.de   + drush -l sub.domain.de cr -y
sub.domain.de    [success] Cache rebuild complete.
 

I am not sure yet if this happens every second time the job is run but I think so. It is not always the same subdomain this happens. To me it looks like the split is set to true in the settings.php but later on in drush cim somehow the context is lost. But, as I said, it only happens when running with jenkins and that stage parameter, not when running locally when the domain is matched like that: preg_match('/local.intern.domain.de|.ddev.local|.ddev.site$/im', getenv('VIRTUAL_HOST'),$matches or when run in production where no stage is set. Meanwhile I disabled the use of gnu parallel but same result.

Any hints, ideas how I could debug this further?

Answer 1

Since we are talking about a multisite environment, a predictable solution is to place a settings.global.php file in the web/sites directory, include it as the last file in the settings.php, and enable the split there.

The trade-off is that this needs to be done for all feature branches. On the other hand, this approach provides the flexibility to turn it on or off per branch.

Here’s an example of how you can configure the config split:

In your settings.php, include the settings.global.php file as the last include:

if (file_exists($app_root . '/sites/settings.global.php')) {
  include $app_root . '/sites/settings.global.php';
}

In your settings.global.php file, you can enable the config split like this:

$config['config_split.config_split.config_dev']['status'] = TRUE;