I'm trying to index Search API Solr using Drush 7 on platform environments, but the drush search-api-index command is failing for a very strange reason.
There are clues of the problem below, but I'm not sure what to do next to figure this out. This issue has me very stumped.
1. I've configured platform.sh-specific drushrc.php file, which sets up the --uri option correctly.
web@server:~/www$ drush st
Drupal version : 7.59
Site URI : https://mysite.us-2.platformsh.site/
Database driver : mysql
Database hostname : database.internal
Database port : 3306
Database username : user
Database name : main
Drupal bootstrap : Successful
Drupal user :
Default theme : mysite_theme
Administration theme : seven
PHP executable : /usr/bin/php
PHP configuration : /etc/php5/cli/php.ini
PHP OS : Linux
Drush script : /app/.global/vendor/drush/drush/drush.php
Drush version : 7.4.0
Drush temp directory : /tmp
Drush configuration : /app/www/sites/all/drush/drushrc.php
Drush alias files : /app/www/sites/all/drush/aliases.drushrc.php
Install profile : mysite
Drupal root : /app/www
Site path : sites/default
File directory path : sites/default/files
Private file directory path : /app/private
Temporary file directory path : /app/tmp
2. Drush DOES list the search api indexes correctly.
web@server:~/www$ drush search-api-list
Id Name Index Server Type Status Limit
1 Default node index default_node_index (none) Content disabled 50
34 Location Search location_search Solr Server (Platform.sh) Content enabled 50
35 Page manager pages page_manager_pages Solr Server (Platform.sh) Page manager enabled 50
36 Site General Search site_general_search Solr Server (Platform.sh) Content enabled 50
37 Bookplates bookplates Solr Server (Platform.sh) Content enabled 50
38 Staff Directory staff_directory Solr Server (Platform.sh) Content disabled 50
web@server:~/www$ drush search-api-status
Id Index % Complete Indexed Total
1 Default node index - 0 0
34 Location Search 0% 0 2187
35 Page manager pages 0% 0 13
36 Site General Search 0% 0 2187
37 Bookplates 0% 0 2199
38 Staff Directory - 0 0
3. HOWEVER, trying to reindex with drush fails
web@server:~/www$ drush search-api-index
The index Default node index is up to date. [ok]
Indexing a maximum number of 2187 items (50 items per batch run) for the index Location Search. [ok]
Command batch-process needs a higher bootstrap level to run - you will need to invoke drush from a more functional Drupal environment to run this [error]
command.
The drush command 'batch-process 19730' could not be executed. [error]
Drush was not able to start (bootstrap) the Drupal database. [error]
Hint: This may occur when Drush is trying to:
* bootstrap a site that has not been installed or does not have a configured database. In this case you can select another site with a working
database setup by specifying the URI to use with the --uri parameter on the command line. See `drush topic docs-aliases` for details.
* connect the database through a socket. The socket file may be wrong or the php-cli may have no access to it in a jailed shell. See
http://drupal.org/node/1428638 for details.
4. A clue as to what is wrong is then given, with the WRONG db creds and site uri!
Drush was attempting to connect to:
Drupal version : 7.59
Site URI : mysite.local
Database driver : mysql
Database hostname : mysite.local
Database port : 3306
Database username : mysite
Database name : mysite
PHP executable : /usr/bin/php
PHP configuration : /etc/php5/cli/php.ini
PHP OS : Linux
Drush script : /app/.global/vendor/drush/drush/drush.php
Drush version : 7.4.0
Drush temp directory : /tmp
Drush configuration : /app/www/sites/all/drush/drushrc.php
Drush alias files : /app/www/sites/all/drush/aliases.drushrc.php
Drupal root : /app/www
Site path : sites/default
5. How are the wrong parameters being passed?
It's almost as if it is passing the wrong parameters or a default drush alias to the index task from somewhere, but where? I've scanned the code for "mysite.local":
web@server:~/www$ grep -r mysite.local .
./sites/all/drush/aliases.drushrc.php:$aliases['mysite.local'] = array(
./sites/all/drush/aliases.drushrc.php: 'uri' => 'mysite.local',
./sites/all/drush/drushrc.php.example:$options['uri'] = 'http://mysite.local';
./sites/all/modules/custom/powell_res/powell_res.module: if ( $_SERVER['HTTP_HOST']=='localhost' || $_SERVER['HTTP_HOST']=='mysite.local') {
./sites/all/modules/custom/rc_res/rc_res.module: if ( $_SERVER['HTTP_HOST']=='localhost' || $_SERVER['HTTP_HOST']=='mysite.local') {
./sites/default/local.settings.php: 'host' => 'mysite.local',
./sites/default/settings.php: 'mysite.local' => 'local.settings.php',
I would think that the only relevant locations from above are ./sites/all/drush/aliases.drushrc.php, ./sites/all/drush/drushrc.php.example, and ./sites/default/local.settings.php. Neither of the later two should be included at runtime from drush cli. The contents of my ./sites/default/settings.php only include local.settings.php if uri = mysite.local. This leaves only aliases.drushrc.php. Could a default alias of @mysite.local somehow be passed under the covers?
FWIW, I also looked for possible hard-coded drush command-specific config options to the search-api-index command placed in another file but couldnt turn up anything useful. There is a ./sites/all/drush/drushrc_shared.php file but this file doesn't appear to be loaded, nor does it contain any relevant command-specific settings or even a --uri option.
6. Debugging with -vvv --debug
From the output below, The first half of the command runs fine, but then at some point after it finds the drush command file for search_api_index, it internally runs another drush command via "Backend invoke", with the wrong settings.
web@server:~/www$ drush search-api-index -vvv --debug
Drush preflight prepare loaded autoloader at [preflight]
/app/.global/vendor/autoload.php [0 sec, 2.6 MB]
Starting Drush preflight. [0 sec, 2.6 MB] [preflight]
Cache HIT cid: 7.4.0-commandfiles-0-2cc326a65c41fdab5fcf2c6cb9fb2c3b [0.01 sec, 2.66 MB] [debug]
Loading drushrc "/app/www/sites/all/drush/drushrc.php" into "drupal" scope. [0.03 sec, 4.66 MB] [bootstrap]
Bootstrap to phase 0. [0.08 sec, 6.65 MB] [bootstrap]
Bootstrap to phase 5. [0.08 sec, 6.65 MB] [bootstrap]
Drush bootstrap phase : bootstrap_drupal_root() [0.09 sec, 7.03 MB] [bootstrap]
Loading drushrc "/app/www/sites/all/drush/drushrc.php" into "drupal" scope. [0.09 sec, 7.03 MB] [bootstrap]
Initialized Drupal 7.59 root directory at /app/www [0.09 sec, 7.03 MB] [bootstrap]
Find command files for phase 1 (max=5) [0.09 sec, 5.06 MB] [debug]
Cache HIT cid: 7.4.0-commandfiles-1-05d40856b788e54ca64b35c99a2c1fd7 [0.09 sec, 5.06 MB] [debug]
Drush bootstrap phase : bootstrap_drupal_site() [0.09 sec, 5.07 MB] [bootstrap]
Initialized Drupal site mysite.us-2.platformsh.site at sites/default [0.09 sec, 5.07 MB] [bootstrap]
Find command files for phase 2 (max=5) [0.09 sec, 5.07 MB] [debug]
Cache HIT cid: 7.4.0-commandfiles-2-f17fc6bdab2f74278df75194ca483893 [0.09 sec, 5.07 MB] [debug]
Drush bootstrap phase : bootstrap_drupal_configuration() [0.1 sec, 5.3 MB] [bootstrap]
Find command files for phase 3 (max=5) [0.1 sec, 5.34 MB] [debug]
sql-query: SELECT 1; [0.1 sec, 5.45 MB] [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_8YlZWK --database=main --host=database.internal --port=3306 --silent < /tmp/drush_WhdxK8
1
sql-query: SHOW TABLES; [0.11 sec, 5.45 MB] [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_I34QoZ --database=main --host=database.internal --port=3306 --silent < /tmp/drush_INSUan
[output redacted for brevity]
Drush bootstrap phase : bootstrap_drupal_database() [0.14 sec, 5.54 MB] [bootstrap]
Successfully connected to the Drupal database. [0.15 sec, 6.07 MB] [bootstrap]
Find command files for phase 4 (max=5) [0.15 sec, 6.07 MB] [debug]
Drush bootstrap phase : bootstrap_drupal_full() [0.15 sec, 6.07 MB] [bootstrap]
Cache HIT cid: 7.4.0-install_profile-66ecfeb9791a023150773849f1550c5d [0.45 sec, 40 MB] [debug]
Find command files for phase 5 (max=5) [0.45 sec, 40 MB] [debug]
Cache HIT cid: 7.4.0-install_profile-66ecfeb9791a023150773849f1550c5d [0.45 sec, 40.01 MB] [debug]
Cache HIT cid: 7.4.0-commandfiles-5-30eb6a40e5b156e214df63a4a8fcfecc [0.45 sec, 40.06 MB] [debug]
Bootstrap to phase 6. [0.51 sec, 44.05 MB] [bootstrap]
Drush bootstrap phase : bootstrap_drupal_login() [0.51 sec, 44.06 MB] [bootstrap]
Find command files for phase 6 (max=6) [0.54 sec, 41.45 MB] [debug]
Found command: search-api-index (commandfile=search_api) [0.54 sec, 41.45 MB] [bootstrap]
Calling hook drush_search_api_index [0.59 sec, 44.46 MB] [debug]
The index Default node index is up to date. [0.61 sec, 45.96 MB] [ok]
Indexing a maximum number of 2187 items (50 items per batch run) for the index Location Search. [0.61 sec, 45.96 MB] [ok]
Cache HIT cid: 7.4.0-alias-path--0e94eb35635a057777b446898cdd620b [0.62 sec, 46.11 MB] [debug]
Backend invoke: /usr/bin/php /app/.global/vendor/drush/drush/drush.php --php=/usr/bin/php --backend=2 --user=0 --verbose --debug --root=/app/www [command]
--uri=mysite.local batch-process 19738 2>&1 [0.62 sec, 46.13 MB]
/usr/bin/php /app/.global/vendor/drush/drush/drush.php --php=/usr/bin/php --backend=2 --user=0 --verbose --debug --root=/app/www [notice]
--uri=mysite.local batch-process 19738 2>&1 [0.62 sec, 46.13 MB]
7. I've found a related issue #1550575, but it claims to have been fixed for 6 years.
8. Drush @self alias is being weird.
If I run drush st the output is correct (see #1 above). If I run drush @self st I get the WRONG db creds and site uri values:
web@server:~/www$ drush @self st
Drupal version : 7.59
Site URI : mysite.local
Database driver : mysql
Database hostname : mysite.local
Database port : 3306
Database username : mysite
Database name : mysite
PHP executable : /usr/bin/php
PHP configuration : /etc/php5/cli/php.ini
PHP OS : Linux
Drush script : /app/.global/vendor/drush/drush/drush.php
Drush version : 7.4.0
Drush temp directory : /tmp
Drush configuration : /app/www/sites/all/drush/drushrc.php
Drush alias files : /app/www/sites/all/drush/aliases.drushrc.php
Drupal root : /app/www
Site path : sites/default
9 drush sa @self randomly picks the wrong alias from aliases.settings.php
web@server:~/www$ drush sa @self
$aliases["mysite.local"] = array (
'root' => '/app/www',
'uri' => 'mysite.local',
'path-aliases' =>
array (
'%dump' => '/app/db/local-20180830051250.sql',
),
'command-specific' =>
array (
'sql-sync' =>
array (
'structure-tables-key' => 'custom',
'structure-tables' =>
array (
'custom' =>
array (
0 => 'cache',
1 => 'cache_block',
2 => 'cache_bootstrap',
3 => 'cache_field',
4 => 'cache_filter',
5 => 'cache_form',
6 => 'cache_image',
7 => 'cache_libraries',
8 => 'cache_menu',
9 => 'cache_page',
10 => 'cache_path',
11 => 'cache_search_api_solr',
12 => 'cache_token',
13 => 'cache_views',
14 => 'cache_views_data',
15 => 'history',
16 => 'sessions',
17 => 'watchdog',
),
),
),
'rsync' =>
array (
'mode' => 'rlptDz',
),
),
'target-command-specific' =>
array (
'sql-sync' =>
array (
'enable' =>
array (
0 => 'stage_file_proxy',
),
),
),
);
I suspected drush was picking the first alias defined from the ./sites/all/drush/aliases.drushrc.php file, but when I added another different alias and changed the order, I confirmed that it does not do this. Thus, I don't understand why it decides to pull this specific one as the @self.
drush @self cmd; it is nonsensical. Aliases in the first position tell Drush which site to bootstrap. The@selfalias is defined to be the site that Drush bootstrapped. Using@selfin the first position is therefore circular. Drush should throw a hard error for this, but does not. You may usedrush sa @selfto see the selected site@selfin real drush commands, but this is exactly what drush 7's backend invoke seems to be doing, so I figured i'd look to see what @self generated (hence item #8). Thanks for tip aboutdrush sa @self-- will add that output as point #9 above.$aliases["self"] = array (and the 'uri' is magically set correctly, but all other lines are pulled from the$aliases["mysite.local"]entry. This is exceedingly odd.