16

Some modules have deïnstallation routines. Which typically remove databasetables for that module, variables from the variable table and locales introduced by that module. These routines live in the .install of that module.

Therefore, they cannot be ran without that module being present. So here are our current steps. My question is: can this be done simpler and more effectively? Say I remove foo_bar module.

  1. In the RCS, prepare a new release, where:
    • All css and theme-overrides that use or build on top of- foo_bar are removed.
    • All css and theme-overrides for modules depending on foo_bar are removed.
  2. Push that release to acceptance. Test the deïnstallation (from admin/modules) with a very recent copy of the production database.
  3. If all goes well, deploy the new codebase to production, and deïnstall foo_bar and its dependencies there. This will invoke the uninstall in the various modules, cleaning up the database.
  4. In the RCS (git), prepare a new release where the code is actually removed.
  5. Deploy that to acceptance where we test if nothing accidentally depended on this (some ugly modules or theme functions include files directly from other modules. Most notably CSS, JS or image-files).
  6. If accepted, deploy new release to production. production now has a clean database and a clean codebase.

Problem that I can't see how to solve, is that this always needs two releases. Since in Drupal a release requires the site to be offline, this means two times downtime just to remove one module. It also requires two release procedures, which, in professional hosting environments can be very expensive, time-consuming or frustrating.

If we remove the module from codebase in the first iteration, we cannot run the uninstall hooks, keeping many lint in the database; not just a few tables, but mostly variables and locales. If we don't remove the module from codebase, that means the codebase will grow with stale, unused code; this gives no performance overhead, but does make maintaining the code harder and harder.

How do you deal with this?

[edit: added note about deployment being a tough procedure, often]

|improve this question|||||
  • 2
    If you do steps 1 - 6 on a staging server first, couldn't you then update the live site to HEAD^, do the uninstalls, then update to HEAD (all in one sitting)? – Andy Jan 2 '12 at 13:57
  • If all of my projects were git-deployed, then yes. But some need tarballs to be mailed around, while others use (only!) ftp and so on. But looking into git, and some git-hook-scripts is certainly a very interesting idea. – berkes Jan 2 '12 at 16:23
  • Why exactly is it required to take the site down? – Letharion Jan 2 '12 at 22:01
  • @Letharion: 1) Taking the site down prohibits from unwanted writes to your database during the process of altering that database; Drupal does not use Transactions. 2) Deploying new code that depends on certain database state (a theme that requires a certain cck-field, e.g.) will break your site in the time between rolling out code and updating the database. – berkes Jan 3 '12 at 14:37
7
+50

Be very careful about keeping your database and code in sync; as you mention in your question, the modules to be uninstalled need to stay in the code base until their uninstall hooks are run on the live database. This is a limitation of Drupal that a git pull workflow alone is not going to solve.

I would recommend that instead of attempting to adjust your process, you instead look for ways to reduce the downtime required to process your updates. I would recommend setting up a ying / yang multisite staging environment to solve this problem. n.b. I have not used the scripts contained in the preceding link; you may wish to set things up differently, following the same idea that you can swap your live and stage sites during deployment.

Continue to follow the same procedure you outlined in your question with the following adjustments:

a. Sync from dev to stage (yang) as usual. Test by doing an uninstall of the modules to be removed followed by code removal, etc. Git workflow notes: create a tag or note the hashid of the different states of your code: all modules in place prior to uninstall, code modules removed, your overrides & c. removed, etc. as needed. Perhaps only two references are needed.

b. When testing is complete and accepted, restore the code on stage (yang) to the state of live (ying).

c. Prepare the live (ying) site for update by disabling any user's ability to change content on the system. An sql update to the permission table will usually do here. At this point, users will still be able to read content on the live site, but will get a permission denied error if they try to update content. (If you are cool, perhaps you could change the permission denied handler to print an appropriate notice that the function is temporarily unavailable).

d. Now push the live (ying) database back to stage (yang) database, excluding the permissions table from the update.

e. Repeat step a. again. If you have your hashtags handy, it should be easy to restore to the state where the modules to be removed exist, run the uninstall hooks on the database, and then advance back to the state of the code where your items from step 1 are merged back in.

f. You are now ready to swap ying and yang. Do this by adjusting your Apache configuration directives. Note that if you do an /etc/init.d/apache restart, some connections may be dropped, but /etc/init.d/apache reload will allow for a clean swap.

g. Live is now 'yang'; the permissions table is unmodified here, so users can create content. If you automate steps e. and f., the time unavailable should be very low.

h. Push live (yang) back to stage (ying), both code and database -- or push from dev, as needed. You now have a clean environment ready for your next iteration.

|improve this answer|||||
  • Ying-yang fails horribly on step c. Only very specific sites, such as editorial-driven-anon-access-only will work. That is mostly because not only comments, nodes and such must be disabled, but session-table, watchdog, counters, etc will be updated and written to. Downtime seems an unfortunate, but unescapable side-effect. Though, indeed, for certain sites ying-yang is a very interesting concept to use when deploying. – berkes Jan 3 '12 at 14:33
  • Of course you are correct; however, if you script the final step, the window of downtime or lost information will be low. If you lose an entry from the session table, the user will have to log in again. A counter might be off by a little. You might miss a notification or two in watchdog. If this is worse than a short period of "this site is down for maintenance," then by all means use the simpler solution. If you really wanted to, you could attempt to recover the counter and watchdog messages after the swap. This might be more complicated than it was worth, unless info+uptime VERY valuable. – greg_1_anderson Jan 3 '12 at 15:33
  • You might consider keeping track of transactions on the transition site using the mysql binary log. See dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html. I would be reluctant myself to just merge things together, but you could keep track of extra counter / watchdog messages out-of-band. The easiest way to deal with the session table would be to also disable logins during the transition. – greg_1_anderson Jan 3 '12 at 15:40
  • Your comment brought me to a possible other solution: aggregate all the hook_uninstall's for a module as hook_update_n()s of a special purposed module: uninstall.module. That way I can remove the codebase in first iteration /and/ get much faster deïnstallation in the actual release. Could be a drush-script that scrapes this information. – berkes Jan 3 '12 at 20:07
  • 1
    One more thing that will help a little. Change all of your custom php code such that any reference to the module-to-be-removed is wrapped in if (module_exists('removeme')) { ... }. Deploy that code. If you test and confirm that removing the module no longer breaks your custom code, that will simplify your deployment. It is still advisable to do the disable step on a site that is not live, but perhaps this will narrow your window slightly. I don't think that your custom update hook will make it any safer to do a live module disable. – greg_1_anderson Jan 4 '12 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.