There is no single management utility (maybe some managed cloud solutions) which can easily manage your all Drupal instances, so you have to learn how to mix different tools together (e.g. Puppet + drush). Most of it can be done by drush.
Ansible
Ansible is a flexible and extensible automation tool which definetly can help with multisite deployment.
See:
Puppet
Probably you don't need it, but in case you'd need to set-up whole LAMP configuration each time (e.g. on multiple servers), you should use some configuration management utility.
For example to automatize LAMP environment configuration (Linux, Apache, MySQL and PHP) you can use either Puppet or Chef. Having proper Puppet recipe, you're able to configure whole LAMP configuration in one command. Check some basic recipe example for ADS distribution: ads.dev.pp.
Installing Puppet is easy (sudo apt-get -y install puppet), then you've to install necessary modules (e.g. for Apt, Apache and Pear). In example:
sudo puppet module install puppetlabs/apt; sudo puppet module install rafaelfc/pear; sudo puppet module install puppetlabs-apache
For more advanced recipe (different db per vhost), check: Proper way of importing many manifest files with vhost definitions.
Drush remotes
Once you've LAMP environment ready, you need to define your remote shell aliases, so you can transfer your files and database across your sites. Check example.aliases.drushrc.php how to use it.
You can use different remotes for each multi-site instance, e.g.:
// The site in sites/default/
$aliases['site.default'] = array(
'uri' => 'default',
'root' => '/path/to/drupal',
);
// The site in sites/site1/
$aliases['site.site1'] = array(
'uri' => 'site1',
'root' => '/path/to/drupal',
);
Make sure that you'll add your SSH key into your authorized_keys, so it won't ask you for the password each time when running remote commands. See: How to add RSA key to authorized_keys file?.
Site installation
If you'd like to install Drupal sites from scratch, consider using Drupal installation profiles or drush make files (writing one or generating from the existing site).
You can simply check how distributions such as Recruiter, Commerce Kickstart or ADS work.
For complex solutions, you may use phing (or ant). Check example of build.xml from ADS or check Template Phing build.xml file for Drupal projects.
See also: What is the easiest way to install clean Drupal from scratch?
Transfering the files to the remote
Once you have your shell aliases set-up, transferring files is easy.
In example:
drush -v rsync drupal/ @remote
Multi-site example:
drush -v rsync drupal/ @remote1,@remote1
Or this can be achieved by defining each e.g. dev alias in separate *.aliases.drushrc.php group alias files, so calling @dev would invoke all defined dev environments.
See also: How to rsync files between two remotes which share the same host?
Synchronising database between sites
Transfering database is easy as well by drush sql-sync, e.g.:
drush sql-sync @self @remote
This will transfer your local Drupal database into remote. You can also use the same command to transfer the database between two remotes.
Managing settings file
You can consider to edit your settings.php and include at the end extra file such as settings.local.php as your instance-specific configuration file (example 1, 2), then it'll be easier to maintain (as it can be ignored by git). So if you have multiple environments, you can use the following simple approach to load the settings based on your current environment:
$conf['environment'] = $env = 'local'; // dev, test, prod)
And use the simple switch statement to invoke different settings per environment.
Here are the example settings files from ADS distribution which you can use.
If you want to automate changes to your settings file, you can define some drush alias to do something like sed s/DB_NAME/real_db_name/ (drush normally accept arguments and pass them to the remote), or check Can you modify the settings.php file using drush?.
Drush shell aliases
If you want your dream to come true, use drush shell aliases to define your commands which you'd like to execute on remotes as part of your deployment script.
Same examples for your remote:
$aliases['dev'] = array(
'shell-aliases' => array(
'deploy-code' => '!git fetch origin && sudo git stash && sudo git reset origin/dev --hard',
'deploy-db' => '!drush sql-sync --yes @prod @self',
'deploy-files' => '!drush --yes rsync @prod:%files @self:%files',
'deploy-drupal' => "!
sudo -uwww-data drush -y updb &&
sudo -uwww-data drush cc all &&
sudo -uwww-data drush -y fra &&
sudo -uwww-data drush cron &&
sudo -uwww-data drush status-report --severity=2 &&
echo Deployment completed.
",
));
And then you can combine them by defining your local aliases in your drushrc.php, e.g.
$options['shell-aliases'] = array(
'deploy-dev' => '!drush @dev deploy-db && drush @dev deploy-code && drush @dev deploy-files && drush @dev deploy-drupal',
);
So by running simply drush deploy-dev it will transfer the files and database from prod into your testing (dev) environment, run updates, clear cache, run cron, revert all features, etc.
Your use case
Based on above, your use case can be solved by the following commands on local:
Clone the site:
drush -yv @site exec drush rsync sites/site-1 sites/site-2
Upload the settings file and modify the database:
drush rsync examples/default.settings.local.php @site:sites/site-2/settings.local.php
drush -vy @site exec sed -i -e s/DB_NAME/real_db_name/ sites/site-2/settings.local.php
Transfer the db between sites.
drush -vy @site sql-sync --create-db site-1 site-2
Enable chosen set of modules:
drush -vy @site en module_1 module_2
You can create easily the drush alias for that. E.g.
drush clone-site site-1 site-2 # Last 2 are arguments.
Eventually check how to use policy.drush.inc, so you can add your own pre-process, post-process code for any of your drush command.
Read more:
