How to programmatically submit a form that has a file-field using drupal_form_submit()?

Question

I'm working to allow the submission of an existing form via REST using the Services module and drupal_form_submit(). Though this is reasonably routine, the form in question takes a file-upload which is a required field.

To avoid embedding extra logic in the form itself to handle files differently under programmatic submission than under manual submission, is there a trick for getting Drupal to accept a file submitted "headlessly" via REST?

Most of this is not relevant to the answer, but for completeness, here's how I'm set up:

function mymodule_my_resource_create($data) {
    $form_state = array();
    $form_state['values'] = $data + form_state_defaults();
    drupal_form_submit('mymodule_form',$form_state);

    $errors = form_get_errors();
    $status = '200 OK';
    if (!empty($errors)) {
        $status = '400 Bad Request - submission caused errors';
    }
    drupal_add_http_header('status',$status);

    return (object)array(
        'message' => $status,
        'errors' => $errors,
    );
}

function mymodule_form($form,&$form_state) {
    return array(
        'spreadsheet' => array(
            '#type' => 'file',
            '#title' => 'Spreadsheet file',
            // '#required' => 1,
            // ^ there have been issues with this since 4.x...
            // https://www.drupal.org/node/59750
            // so this is done manually in the '#validate' function
        ),
        'reason' => array(
            '#type' => 'textfield',
            '#title' => 'Why file is being submitted',
            '#required' => 1,
        ),
        'submit' => array(
            '#value' => 'Submit',
            '#type' => 'submit',
            '#validate' => array('mymodule_form_validate_upload'),
        ),
    );
}

function mymodule_form_validate_upload($form,&$form_state) {
    if (!isset($_FILES['files']['name'] || !isset($_FILES['files']['name']['spreadsheet'])) {
        form_set_error('spreadsheet','File must be provided');
    }
}

function mymodule_services_resources() {
    return array(
        'my_resource' => array(
            'create' => array(
                'callback' => 'mymodule_my_resource_create'
                'args' => array(
                    'name' => 'post_data',
                    'type' => 'struct',
                    'description' => "Should match contents of $form_state['values'] when submitting through the UI.",
                    'source' => 'data',
                    'optional' => FALSE,
                ),
            ),
        ),
    );
}

function mymodule_actools_plugin_api($owner,$api) {
    if ($owner=='services' && $api=='services') {
        return array(
            'version' => 3,
            'file' => 'mymodule_services.inc',
            'path' => drupal_get_path('module','wos2'),
        );
    }
}

function mymodule_default_services_endpoint() {
    $endpoint = new stdClass();
    $endpoint->disabled = FALSE;
    $endpoint->api_version = 3;
    $endpoint->name = 'my_rest';
    $endpoint->server = 'rest_server';
    $endpoint->path = 'my_rest';
    $endpoint->authentication = array();
    $endpoint->server_settings = array();
    $endpoint->resources = array(
        'my_resource' => array(
            'operations' => array(
                'create' => array(
                    'enabled' => '1',
                ),
            ),
        ),
    );
    $endpoint->debug = 0;
    return array($endpoint);
}

Answer 1

This wasn't as hard as I expected; Drupal has different handling for fields with '#type' => 'file' as opposed to other types. There are two ways of solving the problem. One is simply adjusting the request-payload, expressed here as a curl command:

curl -i \ 
    'https://example.com/drupal/my_rest/my_resource' \
    -H 'Accept: application/json' \
    -H 'Content-Type: multipart/form-data' \
    -F "files[spreadsheet]=@my_spreadsheet.xlsx" # note the weird field-name
# This automatically uses the POST method, so Drupal reads it as a 'create'

Drupal munges file-field-names by default apparently apparently to match the file.inc code. This is stated expressly in comments:

if (!isset($element['#name'])) {
    $name = array_shift($element['#parents']);
    $element['#name'] = $name;
    if ($element['#type'] == 'file') {
        // To make it easier to handle $_FILES in file.inc, we place all
        // file fields in the 'files' array. Also, we do not support
        // nested file names.
        $element['#name'] = 'files[' . $element['#name'] . ']';
    }

If you want to API-endpoint users from the implementation shown above, the other solution is to explicitly provide a '#name' value for the file-field in question, as should be obvious:

return array(
        'spreadsheet' => array(
            '#type' => 'file',
            '#name' => 'spreadsheet_file', // <-- adding this prevents name-munging
            '#title' => 'Spreadsheet file',
            // ...

Note of course that the above might have consequences for other sections of code if the file used to be expected under the old name.