6

A problem I face on a regular basis is trying to test sites that have recently had a lot of module updates. My usual process is to take each module on a case by case basis and test site features related to that module.

For instance, if the wysiwyg module is updated, I'll find a node that implements it in a field and do exploratory testing. But the reality is that the issues can be more subtle - perhaps certain user roles have lost permissions, allowing them to embed media when they were previously restricted.

Sometimes there are so many factors to bear in mind it is impossible to test them all, and invariably the client, at UAT phase, discovers major issues that I may have been blind to for various reasons.

I would be interested to hear of any advice or approaches towards testing sites that have had comprehensive module updates - what can I do to ensure quality, and even, what kind of line should I take with our clients to prepare them for issues that do invariably arise?

5

A solution that I would very much like to test, but haven't so for, is combining a branching model like this one, not necessarily with git, with selenium automated testing. Every night, a tool like Jenkins pulls the development branch, runs drush make, drush site-install, and runs your selenium test suite.

I also remember reading an empirical study about finding bugs in software, that I unfortunately can not find a link to right now. The findings were, that no single way of finding bugs was "good enough", and that at least two separate models, such as "Test-driven", "User beta testing", "Code review", etc, were necessary to reach a high, +75%, level of found bugs, prior to release.

  • I would love to see that study. Part of the battle is winning the team over to automated testing - i.e. getting the budget for it! – persepolis Jan 6 '12 at 10:37
  • 1
    Sorry, spent most of my lunch trying to find it again, but neither google, nor slashdot, where I probably saw the study, helped me find it. If I think of something, I will post back. – Letharion Jan 6 '12 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.