Best practice for moving content from production server to dev server

Question

I have used GIT to build my Drupal 7 site (following these guidelines). In order to move changed content downstream before updating the site – in order not to overwrite new content with old content – I would like to find out what the best practice is for that. I was thinking about using the d2d migration module, but maybe there are more established and proven methods?

How do the professionals do this?

Answer 1

Modifying an existing project without loosing data is a common need.

Use Features

One of the best practice I know, is to use the Features module to deliver all the modifications you have made on the production server. Features allow you to make custom modules containing content type structures, views definitions, and a lot more.

You'll probably also need to use the Strongarm module to be able to retrieve the settings variables of some part (modules configuration for instance).

Those modules really help to deploy modifications on a production site, even using git. You'll find easily some good tutorials about using Features.

The only important part to know is when you start using Features, it's difficult to stop using it. Because it will create and implement custom module containing all the specifications of your website (content types, fields, views, pages, ...). If you disable it / them, everything disappear.

Export modifications

If the modifications on your project are not to complicate (mainly content types structures and views), you can use the export / import possibilities offer by some modules (views for instance).

You can also use Bundle Copy module for the node content types export and import part.

Hard way

After, there is a hard way I use sometimes. It is working only if you did not change the fields settings of your existing content types, and if you just add some (fields and content types).

You can copy in your development server database the content tables of your production server. You have to well know all your content types, and specifically the fields used in them (specially the machine names of the fields).

=> You will lose all the new and modified nodes of your dev server. Then do not hesitate to make a backup of your database first.

If you are in the case, you can copy from the production server all the content tables :

• node : the base node table with all the content informations
• node_revision : if you want to get back the revisions informations
• for each one of your content type fields :
  • the data table : field_data_[field_name] (ie : field_data_body)
  • the revision table : field_revision_[field_name] (ie : field_revision_body)
• url_alias : The custom path of your content (if needed)
• taxonomy tables : if you need to, but it can be tricky
• user tables : users and users_roles (if needed)

Answer 2

Thanks for all your suggestions!

In case anyone else looks for an alternative solution I decided to post this answer to my own question. When the production site needs to be updated with changed theming, new functions, custom modules, new code etc, I do the following for the time being:

1. Using Putty I login at the remote production server and export all nodes from each content type using the Node Export module with Drush (>drush ne-export --type=mycontent_type --file=transfer/mycontent_type.txt).
2. After exporting all nodes from each content type I use FileZilla to transfer the transfer folder with all (8) export files to the dev site.
3. On the dev site I use Drush to import all nodes of each content type. These three steps takes about 10-15 minutes with around one thousand nodes and 8 content types.
4. The production site and the dev site now has the exact same content. If needed I also transfer images from the production sites/default/files to the dev side, which makes it easier to see that everything looks identical.
5. On the dev site I now use Migrate & Backup to backup the database. On my production branch I add, commit and push my local changes to my GitHub repo.
6. Then I login at my staging site with Putty and pull my production branch from Github.
7. On the staging site I restore the database backup from step 5 with Backup & Migrate.
8. If everything looks good I put the production site in Maintenance mode and repeat step 6 and 7 there.

This way I will have a fresh copy of the live content on the dev side (and I can easily move content downstream at any time).

As for the answers provided I will certainly explore the Features module and the other possibilities suggested. Thanks again. And please let me know if you spot a flaw in my current method.

Maybe I should mention that there are only a few people that are permitted to add new content or make changes on the live site. That means that I can see to that no new content changes are made during the update procedure.

Answer 3

Since D7 keeps its configuration in sql, when you sql-sync from production to dev (as described below), any configuration changes made to dev will be overwritten. (This has been fixed in Drupal 8, where configuration is stored separate from the content.)

It is sometimes recommended that one use Features to push database configuration changes made on dev to files outside of the sql database, which then can be file-synced to production and then re-imported into the database. Unfortunately, it is not trivial to retrofit an already existing site to use Features for configuration, and not all modules support Features. While some people report they have success with this method, I have not been able to make it work for my sites - but YMMV.

Instead, the way I do this is to keep a fairly detailed diary of all configuration changes and additions that I make to the dev site. After the sql-sync, I use this diary as a script and manually add these configuration changes back to dev.

Since you're running the same code base on both servers, there is no need to use d2d migration or any other module to migrate the sql database - standard shell tools will do.

Given that the database holding Drupal is named "drupal7":

I first clear all caches (to reduce the size of the dump) and then do a full dump of the "drupal7" DB on my production server:

drush cc all
mysqldump -u root -p drupal7 > productionYYMMDD.sql

I then move the dump file (using scp) from production to dev and insert the full dump on the dev server:

mysql -u root -p drupal7 < productionYYMMDD.sql

After manually adding the configuration changes (using the diary as a script), dev is now ready to be deployed.

Git isn't relevant here, as it is used for syncing code, not content.