Move File Field storage from public:// to another stream wrapper (like s3://)

Question

I've been looking around for this a bit, and it seems like it would be relatively easy in Drupal 7, since D7 supports stream wrappers, and Fields are part of core... but I can't find any documentation.

I have a file field with a few thousand files attached to nodes, and it's using public:// right now.

I also have the AmazonS3 module installed, and some other file fields are already set to use the s3:// stream wrapper.

I now want to migrate the file field using public:// to s3://, and I want all the files already uploaded to the site's files folder to switch to s3. I can manually copy over the files to a bucket on S3, but what exactly would I need to do in Drupal/the database to get the files to be loaded over s3 instead of public?

I thought about just writing a custom query to update all rows in the file_managed table with public://custom/path to change to s3://custom/path, but (a) there are some older files that were migrated with different paths than custom/path, and (b) I'm not sure if this is the right way to get Drupal to use S3.

[Edit: Just noticed the file_move() function in the API—would I be able to move files using this (after switching the field storage in the Field UI) instead of having to manually copy files and update the database?].

Answer 1

For the paths that match across the two servers you can just change the paths in file_managed to start with s3://, clear the caches, and the S3 module will handle the rest.

This module should be really helpful, I've just written a module for someone which is pretty similar to what you're trying to do (except the files are pruned from the local file system to S3 on a cron) and the module in that link was a great starting point.

Since the S3 module implements a fully functional stream wrapper you can use all of the usual file functions (file_copy, file_move, file_delete, etc) as normal, without having to worry about where the files are.