Drush 7 bootstraps DB apparently without error, but still needs higher level

Question

This is a Drupal 7 multi-site (subdirectory based with symlinks) I've migrated to a new server. (Perhaps worth mentioning Drush doesn't work on the old server either). The websites don't have any obvious problems.

Bootstrapping appears to reach level 5 without any error. Debian 9.4, PHP 5.6. I've checked the PHP.ini requirements and I've definitely got the mysql PDO driver.

I originally installed Drush 7.x via Ansible (geerlingguy.drush role). I've now switched to Drush 8, installed manually (downloading Drush.phar from GitHub):

drush status (domains redacted):

(Note - if I use drush status --uri=example.com/subsite1 or drush status --uri=example.com/subsite2, then the 'Drupal Settings File' and 'Site path' lines, plus database name etc. are updated accordingly, so the right settings.php file is being read and there doesn't appear to be a problem switching between sites.)

 Drupal version         :  7.58                       
 Site URI               :  http://default             
 Database driver        :  mysql                      
 Database hostname      :  localhost                  
 Database port          :  3306                       
 Database username      :  redacted            
 Database name          :  umbrella                   
 PHP configuration      :  /etc/php/5.6/cli/php.ini   
 PHP OS                 :  Linux                      
 Drush script           :  /usr/local/bin/drush       
 Drush version          :  8.1.16                     
 Drush temp directory   :  /tmp                       
 Drush configuration    :                             
 Drush alias files      :                             
 Drupal root            :  /var/www/example/htdocs   
 Drupal Settings File   :  sites/default/settings.php 
 Site path              :  sites/default              

Note for others: if Drush can't ever find the settings file, it'll display:

Drupal Settings File   :  MISSING

drush up --debug

Phar detected. Proceeding to drush_main().
Cache HIT cid: 8.1.16-commandfiles-0-dbfc5eac52981945177b460bd5c08987 [0.01 sec, 3.45 MB]                                                                                                                                                                     [debug]
Bootstrap to phase 0. [0.14 sec, 9.3 MB]                                                                                                                                                                                                                  [bootstrap]
Bootstrap to phase 6. [0.14 sec, 9.31 MB]                                                                                                                                                                                                                 [bootstrap]
Drush bootstrap phase : bootstrap_drupal_root() [0.14 sec, 9.69 MB]                                                                                                                                                                                       [bootstrap]
Initialized Drupal 7.58 root directory at /var/www/example/htdocs [0.14 sec, 9.69 MB]                                                                                                                                                                    [bootstrap]
Find command files for phase 1 (max=6) [0.15 sec, 7.3 MB]                                                                                                                                                                                                     [debug]
Cache HIT cid: 8.1.16-commandfiles-1-3f59809f319dc8415f75a4fe9a39c2e1 [0.15 sec, 7.3 MB]                                                                                                                                                                      [debug]
Cache HIT cid: 8.1.16-annotationfiles-1-f59a1308e828ee8afb57dc2e531b4a8b [0.15 sec, 7.3 MB]                                                                                                                                                                   [debug]
Drush bootstrap phase : bootstrap_drupal_site() [0.18 sec, 7.33 MB]                                                                                                                                                                                       [bootstrap]
Initialized Drupal site default at sites/default [0.18 sec, 7.33 MB]                                                                                                                                                                                      [bootstrap]
Find command files for phase 2 (max=6) [0.18 sec, 7.33 MB]                                                                                                                                                                                                    [debug]
Drush bootstrap phase : bootstrap_drupal_configuration() [0.18 sec, 7.33 MB]                                                                                                                                                                              [bootstrap]
Find command files for phase 3 (max=6) [0.19 sec, 7.36 MB]                                                                                                                                                                                                    [debug]
sql-query: SELECT 1; [0.19 sec, 7.48 MB]                                                                                                                                                                                                                     [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_R3Phpd --database=umbrella --host=localhost --port=3306 --silent  < /tmp/drush_T5CkBa
  1
sql-query: SHOW TABLES; [0.2 sec, 7.48 MB]                                                                                                                                                                                                                   [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_XA8Y3i --database=umbrella --host=localhost --port=3306 --silent  < /tmp/drush_lIZBeg
  actions
  apachesolr_environment
  apachesolr_environment_variable
  apachesolr_index_bundles
  apachesolr_index_entities
  apachesolr_index_entities_node

    [... long list of tables ...]

  webform_emails
  webform_last_download
  webform_localization
  webform_roles
  webform_submissions
  webform_submitted_data
  wysiwyg
  wysiwyg_user
Bootstrap to phase 1. [0.22 sec, 7.57 MB]                                                                                                                                                                                                                 [bootstrap]
Bootstrap to phase 6. [0.26 sec, 9.96 MB]                                                                                                                                                                                                                 [bootstrap]
Bootstrap to phase 5. [0.26 sec, 9.96 MB]                                                                                                                                                                                                                 [bootstrap]
Command pm-update needs a higher bootstrap level to run - you will need to invoke drush from a more functional Drupal environment to run this command. [0.26 sec, 9.97 MB]                                                                                [error]
The drush command 'up' could not be executed. [0.26 sec, 9.97 MB] 

If it's apparently connecting to the DB OK, what else should I be checking?

(Thanks.)

Answer 1

Looking at the output again, I notice that Drush reports that the site "default" is initialized. Could it be as simple as helping Drush to select the correct site using the URI option - ref http://docs.drush.org/en/8.x/usage/

Answer 2

This is caused by using database prefixes to share user/permission tables between multiple sites (turns out Drush has never really supported that).

There are a number of stages in the database bootstrapping phase. Although drush_valid_db_credentials() succeeds (and hence you can use drush sql- commands without issue), bootstrap_drupal_database_has_table() subsequently fails, because it's trying to look for a table named like this - "database.table" among a list of tables without any database prefixes.

That list of tables shown in my debug output is actually as part of that test, and the phase fails immediately, it's just not visible in the debug info.

These bootstrap messages…

Bootstrap to phase 1. [0.22 sec, 7.57 MB]                                                                                                                                                                              
Bootstrap to phase 6. [0.26 sec, 9.96 MB]
Bootstrap to phase 5. [0.26 sec, 9.96 MB]  

…only refer to the start of the process - they're not telling you it was completed successfully. In fact what's happening is more like this (with extra debugging added):

Bootstrap to phase 1. [0.85 sec, 8.77 MB]
result for 0 = 1
result for 1 = 1
Bootstrap to phase 6. [0.92 sec, 11.11 MB]
result for 0 = 1
result for 1 = 1
result for 2 = 1
result for 3 = 1
fail for 4
Bootstrap to phase 5. [0.92 sec, 11.11 MB]
result for 0 = 1
result for 1 = 1
result for 2 = 1
result for 3 = 1
fail for 4

result = 1 indicates success. Note each phase only actually runs once - the results are cached in a static array to speed things up.

Workaround

With this I can still use Drush, and I don't need to modify the Drush source code, just add a conditional to each settings.php:

if (defined('DRUSH_VERSION')) {
    // Drush doesn't construct prefixes correctly, remove and use single DB
        // remember to call Drush with correct --uri= value!
    $db_prefixes = [];
} else {
    $db_prefixes = // whatever they usually are
}

$databases = [
...

unset($db_prefixes);

We use the Drush constant to detect if Drush is running or it's a webpage. Doing it this way, rather than, for example, checking for the presence of $_SERVER variables, also ensures things work properly if Drupal code is executed from the command line in other ways without Drush.

You then call Drush and user --uri to specify the appropriate settings.php:

drush up --uri=www.example.com/
drush up --uri=www.example.com/subsite

It's important to remember to use the correct site when you want to edit anything to do with users or roles (i.e. to ensure you're editing the correct set of MySQL tables).

See also:

• GitHub issue - bootstrapping
• GitHub issue - Drush debugging