Server Configuration causes file uploads to fail

Question

I have a drupal 7 site running on a Redhat shared hosting environment that uses zend php. I am having problems with file upload on content types and webforms that accepts various file types up to 300MB. I have modified the server's php upload limit through the php.ini file, and can confirm that the limit is raised through phpinfo() and the drupal status report that post post_max_size and upload_max_filesize. I believe this is a server configuration error because the site functions as expected on both my ubuntu development machine and test server, so the only possibility is with the settings on the shared redhat server environment.

The errors occur when I try to upload files in the 30-80 MB range, either to my content type or through the webform. In either case when a file is uploaded the request takes an abnormally long time and then fails with the following errors:

An AJAX HTTP request terminated abnormally.
Debugging information follows.
Path: /file/ajax/field_training_materials/und/form-_TZM0uXFQeo35YReaxuUjeHprp2tr192kNHTucXffSM
StatusText: n/a
ResponseText: 
Error  
The website encountered an unexpected error. Please try again later.    

ReadyState: undefined

Once, I received the following JS error while attempting the uploads: "Blocked Script Execution in 'node/722/edit' because the document's frame is sandboxed and the 'allow-scripts' permission is not set" and "Blocked form submission to '/node/722/edit' because the form's frame is sandboxed and the 'allow-forms' permission is not set. Googling these errors didn't turn up any useful information.

From what I can tell the files get almost completely uploaded to 57 of 58 MB and is placed in the correct directory without being attached to the content type. It seems to mostly have problems with audio files, but I don't have a range of different file types at different sizes to test for a pattern of breakpoint.

I researching the problem, I read that some people have trouble with the Suhosin PHP extension putting extra constrictions on PHP settings, but my live server doesn't have Suhosin running on it. Reading about the Suhosin issue, I have also tried upping the max_input_vars value, which only changed the error to the following.

An unrecoverable error occurred. The uploaded file likely exceeded the maximum file size (300 MB) that this server supports.

From running phpinfo(), I can see that all my php value settings are getting applied, but I suspect that maybe they don't take effect. Since the first error, the error has been failing silently, and nothing meaningful gets placed in my logs. I have also checked that all my php ini_set statements are consistent between .htaccess, settings.php, and the php.ini files.

I have just spoken with tech support the following error was thrown when I tried the file upload: [Thu Feb 02 07:29:10.729972 2017] [core:error] [pid 475229] [client 178.203.233.17:6690] Script timed out before returning headers: index.php, referer:path to my content edit page. I also do not get anything meaningful in the watch dog logs.

The only difference between the file upload attempts with the content type and the webform is that the ajax error's path is different and I get the following JS error in the console: Error: Syntax error, unrecognized expression: .error, #overlay-context=form/training-suggestions-

So, the only thing I am sure of at this point is that it must be a server configuration error. My suspicion is that somehow the the php values settings are getting applied, but perhaps overwritten at some stage. Any advice on how to solve or diagnose this would be greatly appreciated.

Update: So, while trying this again I got a different ajax error, saying the mysql db went away:

   An AJAX HTTP request terminated abnormally.
    Debugging information follows.
    Path: /file/ajax/field_training_materials/und/form-66R_kY340Ob1IV3M9bRKvHevXAdePEElDr6cgwVf_DE
    StatusText: n/a
    ResponseText: Additional uncaught exception thrown while handling exception.OriginalPDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT nt.*
    FROM 
    {node_type} nt
    WHERE  (disabled = :db_condition_placeholder_0) 
    ORDER BY nt.type ASC; Array
    (
    [:db_condition_placeholder_0] => 0
    )
    in _node_types_build() (line 739 of /home/phislub9/public_html/modules/node/node.module).AdditionalPDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT 1 AS expression
    FROM 
    {variable} variable
    WHERE ( (name = :db_condition_placeholder_0) ); Array
    (
    [:db_condition_placeholder_0] => drupal_css_cache_files
    )
    in variable_set() (line 1240 of /home/phislub9/public_html/includes/bootstrap.inc).Uncaught exception thrown in session handler.PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT 1 AS expression
    FROM 
    {sessions} sessions
    WHERE ( (sid = :db_condition_placeholder_0) AND (ssid = :db_condition_placeholder_1) ); Array
    (
    [:db_condition_placeholder_0] => 1A5hUf8pjc2OA0iW8Nlom8qeFgG8rJPPKXwxBKKANBI
    [:db_condition_placeholder_1] => 
    )
    in _drupal_session_write() (line 209 of /home/phislub9/public_html/includes/session.inc).
    ReadyState: undefined

Update: So, I've done some poking around in response to the first answer below. Here's the result of SHOW VARIABLES; in the mysql shell max_allowed_packet | 268435456

Update2: Here's the output

mysql> show global status like 'com_kill';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    3204869
Current database: *** NONE ***

+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Com_kill      | 0     |
+---------------+-------+
1 row in set (0.01 sec)

Update: Checked with hosting tech support, apparently there is unlimited disk usage for website files, however the server reports that the /home folder is at 94%. This was due to serveral large zip files from the previous site maintainer/developer. I've backed up and removed these files, but it hasn't affected the file upload problem or the server's report on the size of the home folder. The only other thing I got from tech support was that the php values for the server MAYBE can't be overridden, or can't be set above certain upper bounds, but the support agent wasn't able to give me definitive answers on what those values were. I've followed up with a support ticket to get better answers to what's going on.

Answer 1

Original answer

Try this for your second, new error:

General error: 2006 MySQL server has gone away

You can try to fix this in two ways:

1. my.ini

Open your my.ini file and locate max_allowed_packet. By default this is 1M. Setting it to 16M and restart your MySQL server. If the problem persists, try setting it to 512M and restart your MySQL server.

2. MySQL prompt

Enter your MySQL prompt, log in as user root or as a user with the SUPER privilege. Run the following command:

SET GLOBAL max_allowed_packet = 1024 * 1024 * 512;

This method doesn't require you to restart the server.

Update

Since this doesn't seem to solve your problem, it might be possible that your SQL statement was killed. Some systems will proactively kill SQL statements that have been running too long. You can easily check if this may be happening proactively by looking at how many KILL statements have been executed.

What does the following MySQL command return?

show global status like 'com_kill';

Answer 2

So, Wim Mostrey's answers were right on point and would have solved the problem in a normal hosting environment. The issue came down to conflicts with Apache's modsecurity settings on the shared host as well as PHP's memory allocation per user on the shared host, taking longer than the MySQL wait time out setting allowed. It took ca. 70 seconds for the php script to write the files to disk and Mysql's wait timeout was only 30 seconds. It wasn't possible to see what was going on clearly until I addressed some other issues that were throwing warnings. In the end the problem was addressed by disabling modsecurity for our server and configuring the Mysql Wait Timeout Module. Hope this helps someone.