Amazon S3 for Stage / Production only?

Question

Is it possible to use the Amazon S3 module for stage / production only? The site I'm building, is using features. So when I export my fields for the content types, I either have the option to select Local File Storage, or the S3 File System.

I thought I had a solution, where on stage / prod, I would hide the local file system wrapper, but the issue turns out, that unless I select the S3 checkbox when editing a field in the content type, it doesn't default to S3.

I could go thru and select this option for all file fields, but then on the downside, dev will be broken.

My only other option, which I haven't tried, is setting a global variable via settings, and then using a if/else logic in the features for each file field, to pre-set the upload location for that field.

Ahhh, so messy.

Anyone come up with a clever solution in the pasts? That allows me to force files to be uploaded to S3 for stage/prod and local file system when running via local or dev?

Update:

Okay, so I noticed that features has the following hook:

hook_field_default_field_bases_alter()

So, in my module, I'm using that hook, determining if the env should be serving the files via local storage, or s3. If it's s3, I then alter it from 'public' to 's3'

I went and just to be safe, re-exported all my features, and made sure that by default, it was set to public, an not s3. Now, all my env's are using the correct storage location. I'm still not sold that what I'm doing is the best solution, but it is working. Any thoughts?

$use_s3f3 = variable_get('image_save_s3f3', FALSE);

// Only on stage/uat/prod, do we want to use s3 as the file storage engine.
if ($use_s3f3 === TRUE) {
 foreach($fields as $key => $item) {
   if (isset($item['settings']['uri_scheme']) && $item['settings']['uri_scheme'] === 'public') {
     $fields[$key]['settings']['uri_scheme'] = 's3';
   }
 }
}

Answer 1

I made a similar patch to the Amazon S3 module: "New variable to override file field URI scheme per environment" https://www.drupal.org/node/2422977#comment-9602981

This patch was adopted by the module, and is now the approved way of solving the OP's problem using the Drupal 7 Amazon S3 module. This is the issue I wrote on Drupal.org.

Problem/Motivation

When using Features (or hook_field_default_field_bases() directly) to export file field definitions, the Upload destination is included. This can already be handled in a custom module by implementing hook_field_default_field_bases_alter(), and checking $conf (set in env-specific settings files, etc). But it would be nice to include support directly in amazons3.module.

Proposed resolution

Implement hook_field_default_field_bases_alter(), and check for a namespaced variable amazons3_file_uri_scheme_override. This variable can be added to environment-specific sites/*/settings*.php files to allow certain environments to upload to S3 while other environments upload to whatever the defined/exported (public or private) URI scheme is.

User interface changes

The only change in the UI would be that the field settings reflect the URI scheme defined either by the exported field bases or the variable (on environments where that's present).

API changes

No API changes, but the patch documents usage of the variable.

Note the module's README.md also now contains this additional line:

When using Features to export field definitions, the Upload destination is included. If you want to override this (for example, in a multi-environment workflow), use the 'amazons3_file_uri_scheme_override' variable. See amazons3_field_default_field_bases_alter() for documentation.

Here is the relevant documentation of amazons3_field_default_field_bases_alter that README note is referring to:

* Allows a variable to override all exported field bases to use 'Amazon S3' as
* the Upload destination. For example this can be added to environment-specific
* Drupal settings files, to allow certain environments to upload to S3 while
* other environments upload to the exported (public or private) URI scheme:
* @code
* $conf['amazons3_file_uri_scheme_override'] = 's3';
* @endcode

Also note the variable is not deleted by amazons3_uninstall() because it's only valid use case is being defined in $conf (otherwise just change the field setting).

Answer 2

You could create a local config for s3 in your local settings.php ?

Instead of trying to change storage types for local dev, you could use s3 locally but on a temporary bucket.

We use a separate bucket for local development that we can trash or whatever, we override all the settings in the conf array;

$conf['s3fs_bucket'] = 'our-dev-bucket';
$conf['s3fs_region'] = 'ap-southeast-2';
$conf['s3fs_awssdk2_access_key'] = 'Access';
$conf['s3fs_awssdk2_secret_key'] = 'Secret';
$conf['s3fs_use_instance_profile'] = FALSE;

Will this work for you? It works in my small team (5 of us.)