I would like to improve my testing environment automation in order to reduce effort needed for quality assurance tasks.
I heard about Phing and Jenkins and read overview information but still I am not convinced it is worth investing the set up effort currently.
Can you explain the costs and benefits of installing Phing, Jenkins based continous integration environment while I am using Drupal, Drush, Git, Simpletest and bash scripts already in dev, staging, production set up ?
Ok, as for being a fan myself of continuous integration, I can assure you virtualisation is very handy to control the testing environment. I have built myself a CI setup, forking off from a wonderful package called quickstart. This package depends on Virtual-Box or other VM software, it is basically a Ubuntu made for Drupal with a pre-set up lamp architecture, pre-configured debugging and editors, profilers, etc.
I have installed Jenkins and phing on top of quickstart, and can assure you it is quite low cost($0), quite low hassle. Any one can co-opt in with me on most any project quite easily, the whole dev environnement is reproduced and it comes with quite a few tools by default, and quite a few I have myself upgraded.
Now, the way we use jenkins is most certainly not pure CI, it just run the tests when we commit, then we shut down the thing and forget about it. It could be used for notifications and documentation generation, and on some projects I do generate some code metrics but the main thing I also cared about in my setup was testing. One clear advantage when using a vm software for testing is it can easily be reproduced used to run the tests elsewhere or even on different systems and configurations, you always have control over the environnement (memory, processor usage, peripheral ressources, etc..)
In most projects I contributed on, testing is done before staging I guess this may be different depending on the project managers.
PS: Another thing to consider, in a true testing environnment you would tend to try and reproduce your server setup and load as much as possible. Our projects are hosted on Cent-OS and obviously we don`t quite follow up to the rules on this aspect either, quickastart is Ubuntu based. So far have not had any problems, and as a bonus has many other features pre-configured for rapid Drupal application prototyping and developpment.
Happy CI'ing!
Where i am working Jenkins is configured and it is very useful.
When the code goes on the development server or live, the coder report will execute and will show all the coding reports by mail. Every user has their login so you can easily identify who added the buggy code.
Coder is one example you can run any number of events on that time as running your testcases when code goes up or run a selenium script.
