While moving a bunch of legacy D7 sites to a new server, I bumped on the following problem with one site only. This is certainly NOT the classical permissions issue on files directory.
Bug description
The images derivatives are removed by some process. Accessing the website in maintenance mode regenerates all the image derivatives, but as soon as I disable maintenance mode and visit a couple of pages, either with admin role or not, the whole styles directory is flushed, leaving sites/default/files/styles
totally empty (!) and no images are regenerated anymore.
This behaves only on one site, other D7 or D8 sites are fine on the same server.
In maintenance mode, set permissions properly, then list styles
ls -ld sites/default/files/styles/*/
drwxrws--- 3 kojo www-data 4096 Jul 2 2016 sites/default/files/styles/gallery/
drwxrws--- 3 kojo www-data 4096 Dec 1 11:15 sites/default/files/styles/medium/
drwxrws--- 3 kojo www-data 4096 Dec 1 11:13 sites/default/files/styles/product/
#... and so on
List again styles after leaving maintenance mode
ls -ld sites/default/files/styles/*/
drwxrws--- 3 kojo www-data 4096 Jul 2 2016 sites/default/files/styles/gallery/
Only that gallery directory was not impacted ; it is indeed not a referenced style in UI, and while I don't remember why it is there _ hum hum old projects :-( _ better ignore it for now. All the styles directories were completely removed.
List after returning to maintenance mode
ls -ld sites/default/files/styles/*/
drwxrws--- 3 www-data www-data 4096 Jul 2 2016 sites/default/files/styles/gallery/
drwxr-sr-x 3 www-data www-data 4096 Dec 1 11:15 sites/default/files/styles/medium/
drwxr-sr-x 3 www-data www-data 4096 Dec 1 11:13 sites/default/files/styles/product/
#... and so on
The whole tree was recreated, with different perms.
Files directory permissions
I did set the same permissions as in any of my D7 sites :
cd /path_to_drupal_installation
chown -R kojo:www-data .
find . -type d -exec chmod u=rwx,g=rx,o= '{}' \; # 750 on directories by default
find . -type f -exec chmod u=rw,g=r,o= '{}' \; # 640 on files (except settings.php 440)
find sites/default/files -type d -exec chmod u=rwx,g=rwx,o= '{}' \; #770 for files dir
I then tried a more permissive way with 775 permissions, just to make sure. No luck.
What else I tried to debug
drush rq
, nginx error log, the server website log, watchdog... return no errors.drush image-flush --all
flushes styles when there are some, but does not regenerate on the fly as described in doc.- Visiting each of the styles in
/admin/config/media/image-styles
, be in maintenance mode or not, recreates the directory tree and the "balloons examples images". - In the evening, I installed ImageInfo Cache and performed a bulk regeneration with
drush image-generate all all
. This worked, and four hours later the styles were still there, but in the morning they had been removed! No CRON task was run during that night ! - Switching from PHP 7.3 to 7.1 or 7.2 has no impact.
- Forcing Varnish to fetch backend does not change behavior.
- I did set settings.php, vhost of this site to be as strictly similar as possible to the other websites on that server.
Environment
For all the moved websites, core is 7.75 and all the modules are up to date.
I'm running them on Debian 9 + NGINX + PHP 7.3. NGINX as proxy for SSL requests, forwarding them to Varnish which upon cache logic, serves cached pages or fetch NGINX back.
By the way I can't reproduce the issue on localhost in a DDEV container whichever PHP version so this might be a prod env problem ?
Summary
- the D7 websites with same core work fine in that environment, with same permissions, settings, vhost
- I can't reproduce the bug in localhost DDEV container (so this should not be related to the theme or some updated module of this specific site ?)
- permissions are set correctly and giving permissive one has no effect
- the system is able to create the default images styles examples (balloons...)
- switching from HTTPS to HTTP has no effect
- changing PHP version has no effect
Questions
Am I missing or misunderstanding a point ?
The key to understand what succeeds is probably this behavior whenever toggling maintenance mode on/off. What at the core level is succeeding then ? And what is in charge of flushing styles and when ?
Any tip or analysis is welcome to help me solving this weird issue. Thanks a lot!