How do I manage collaborative development?

Question

I work with another developer on a Drupal site. We have struggled to find a good way to work on different parts of the site at the same time without getting in each others way. We've tried working on the same development instance of the site, but we often step on each others toes, or bring down the site with some bad code making it impossible for the other to continue working until it's resolved. So we've moved to separate development instances. But now it's a big pain to merge our work into a single instance of the site. We basically end up redoing everything on a shared copy.

The biggest problem we have now is how do we merge database changes, and how do we include the database in our source control system? The files are easy, just track them all (we use git) and merge our work, resolving conflicts where needed. But this doesn't really work with the database. We can take a SQL dump and include that in our git repository, but we can't really merge the databases. The Features module helps a little, letting us export some of our database work into code which can then be versioned and merged. However, not even close to everything supports Features. So...

• What steps can we take to easily merge our database changes?

• How should we version the database (is putting a dump file in git a good way to do it)?

• Are there any modules available that help with some of these problems?

• Or, are we stuck with working on the same copy of the site? (please so no)

---

Edit: In the comments we discussed what things can't be exported with Features and one of them was Taxonomies. There's another question that deals with that.

Answer 1

It is a workflow change but you should get used to working on a fresh dump of the live DB. There are three ways to get changes into the DB.

1. Features. This won't work for everything but will for a lot of things you need.
2. update hooks. When features won't work you can hard code things into an update hook of a module you own.
3. Manual changes. Use sparingly. Some things don't come naturally to features or update hooks and are just much easier to do manually. This is a last resort but sometimes is the only piratical way.

If you can. Several times a day get a fresh dump and test your build, you should have less integration problems.

Answer 2

I answered a similar question and am going to adjust it slightly to answer your question here.

My root suggestion is that you have a development/staging server where code changes are checked out using a Continuous integration system on a frequent basis (e.g., every 5 minutes). Thus, on your local machine, you only work on one feature request/bug report at a time, being sure to clearly delineate this task from others that people might be working on and communicating to your teammates that you are working on it (redmine or other bug tracking is great for this). Then, you commit changes on a regular basis, and they get pulled down to the dev/staging server, as do your teammates. Ideally, you have unit tests built into your continuous integration system (highly recommend luntbuild or QuickBuild for this by the way, but Hudson also works). The CI system or tests can automatically pick up on any conflicts you might have introduced as soon as you check in your code. If you need to make content (non-code) changes, you do so on the dev/staging server.

As to the database portion, I've adopted basically two schools of thought here (a 3rd school of thought, doing database diffs, I won't discuss because the complexity is quite high).

1. Deploy by dropping the production database, and importing a mysqldump of the development database. Optionally, run a regex find/replace beforehand on any hard-coded absolute links which reference the dev URL in the SQL dump. After importing the dev db into prod, automatically run SQL statements (usually via script) afterwards to change any settings that are different for prod than dev (e.g., maybe you have in variables table some connection settings for connecting to external systems that you need to change to point at prod external systems instead of at the dev version).

2. Use the Features module, as mentioned by budda, for admin settings, and use the Node Export module for content export/import in combination with the Delete All module. So workflow is:

use node_export and features to export nodes/features to files Optionally (and hopefully) version control Load files on prod system Use drush or admin interface to load features Use drush delete-all or admin interface to delete all nodes of the types you want to import Use drush ne-import or the admin interface to import the nodes from the nodes file you exported. One note, I would highly suggest adopting a standard workflow, where content goes one direction only. Either Dev -> Prod or Prod -> Dev (I prefer this one).

I've done this, and am doing this on some big systems, with fairly good results, but there will always be many ways to slice this apple, choose whichever way works best for you.

Answer 3

While this is an old question with an accepted answer, I believe that there is still room for another one.

First, let me say up front that I don't think Features is the right tool for this task, and shall propose an alternative set of tools.

A prerequisite for team collaboration is to have a staging server for testing out development versions of the project that is separate from your production server. All devlopment code is tested at the staging server, and only pushed to production server when it is stable and ready for deployment. However, the developers doesn't work directly at the staging server. Each developer work at his or her own workstation, using a revision control and source code management (SCM) to coordinate his or her work with the rest of the team.

The SCM system allow team members to work in parallel on different branches of the code without interfering with each other. Only the master branch is deployed on the staging server for testing purposes.

To mirror the database between production, staging and workstations, there is a module named Backup and migrate that can be used if you're on shared hosting and not managing your own database. If you're managing your own database server, this is the only project on that server, and you use mysql, the following pair of commands are handy:

To dump:

mysqldump --all-databases --opt -u root -p > DUMP.sql

To restore:

mysql -u root -p < DUMP.sql

If yours is not the only database on that server, script some version of mysqldump (or equivalent if you're not using mysql) that dumps your databases only.

Make a policy that it is the database on the production server that is master. The staging server and workstations should be a copy of the production database, not vice versa.

Note that Drupal 7 keeps all its admin setting in the database. This means that mirroring the database between the production site, staging site and workstations will migrate admim settings without Features.

Now, for sharing the code:

The standard way for sharing code among the members of a development team is to use SCM system. Drupal happens to be default be managed with such a system named git.

Git allows the use of local or remote repositories. If the team members are located in the same physical space, you can set up a local repository on your staging server. If they're spread geographically, you can set up a remote repositiory. If you don't mind others having read access at your code under development, you can use a sandbox at Drupal.org as a remote repository. You can also use a project area on GitHub. GitHub is not only a repository, but comes with a some tools for collaboration, and allows both public and private repositories.

Basically, a SCM system allow team members to pull source code and documentation from the respository shared by the team members, and push it back in again after having worked on it. The SCM keeps track of changes and if there is a conflict (i.e. somebody tries to push code that doesn't the contain the changes another team member has committed), it will tell you and also suggest a way to resolve this conflict.

Usually, with some cordial communication about how the tasks are divided among the members of the team, there will be no conflicts. But with the SCM system keeping track of things, conflicts become manageable even if mistakes are made or communication fails.

There exists a lot of tutorials on getting started with and using git (GIYF). Two I will recommend are: the git-scm website and Pro Git by Scott Chacon.