I was asked to help with a Drupal 7 site that I've helped to tweak several times before. The problem is, most of the site pages are not shown and "Internal Server Error" 500 page is shown instead (it started at the 6th of may). There are some exceptions though:

/user  (but not /user/1 for example)
/admin/modules  (but not  /admin/config or /admin/reports)

(may be these are not all the exceptions, but I don't know others yet)

Now, here's what I know (I don't have full root access, but I can request access for some resources):

  • PHPMyAdmin shows over 10K requests per minute (84% are select, 6% delete, 6% insert, ..). By the way, it also shows that it was started at the 7th of May (may be administrator has rebooted it)

  • httpd access_log consists of lines like (I splitted into 2 for readability)

    ::1 - - [13/May/2020:02:07:32 +0300] "OPTIONS * HTTP/1.0" 200 - "-"
     "Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_fcgid/2.3.9 PHP/5.4.16 (internal dummy connection)"

    there's nothing except these ones (but I did some attempts to open the site during that period); these ones are added with frequency of about 5-20 lines per second

  • httpd error_log consists mostly of lines like

    [Wed May 13 09:16:09.487641 2020] [fcgid:warn] [pid 27860] mod_fcgid: process 25947 graceful kill fail, sending SIGKILL

    and these are added even less frequently: about 10 times per hour

  • processor is highly loaded: it varies from 10% to 75% CPU consumed by /usr/bin/php-cgi php (which can be seen by using /usr/bin/php-cgi php; usage is increased when I try to load a page, I've seen it bumping from 10% to 50% in my manual tests)

Looks like something's wrong with mod_fcgid, but I have very little idea of what I can do to debug and fix this. Can anybody help with this?

  • If you still need help, Additional information request. RAM size, # cores, any SSD or NVME devices on MySQL Host server? Post on pastebin.com and share the links. From your SSH login root, Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) SHOW FULL PROCESSLIST; AND Optional very helpful information, if available includes - htop OR top for most active apps, ulimit -a for a Linux/Unix list of limits, iostat -xm 5 3 for IOPS by device and core/cpu count, for server workload tuning analysis to provide suggestions. – Wilson Hauck Aug 20 '20 at 22:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.