How to ensure successful update after system failure during updb causing the site to remain in maintenance mode?

Question

Running drush updb to upgrade drupal core or other modules, puts the site in maintenance mode. I recently faced an issue (probably an edge case) where the site was stuck in maintenance mode after running updb in a pipeline of a containerized infrastructure. Here is the sequence of events:

1. Run drush updb -y
2. Container terminated (system failure or unknown reason)
3. Re-attempt at drush updb
4. drupal returns "no updates were required" (this tells me that the step 1 may have completed the update in the background?"
5. Site remains in maintenance mode.
6. I manually bring the site back online.

After all this it looked like the update was successful.

How do I ensure the integrity of the system when this happens without restoring a backup and running the updates again? When running subsequent drush updb that says "no updates required" does this guarantee the previous updates were successful.

Answer 1

When running subsequent drush updb that says "no updates required" does this guarantee the previous updates were successful.

When Drush queues up batch items to be processed for drush updb, it adds these operations:

• Any module schema updates
• Any entity definition updates (off by default, not supported after 8.7)
• A cache clear & post hook update functions (on by default)
• Toggle off maintenance mode (if it not already enabled)

If the only updates were module schema updates then you might be in a scenario where things are ok; there are no post-update hooks needed (e.g. some batch data ops following a schema change) and only the last step of toggling off maintenance mode didn't complete.

How do I ensure the integrity of the system when this happens without restoring a backup and running the updates again?

You'd have to audit the list of updated modules to double-check each schema update was applied and there were no post-update hooks missed in the update.

Depending on your update scenario this may or may not be easy, hence the best practice is doing a DB rollback and re-applying updates to ensure data integrity when uncertain.