Integrating Test Swarm with Hudson
Hudson if you're not familiar with it is a continuous integration server for building and testing software projects.
We're using QUnit as our test runner but Test Swarm supports several test runners out of the box and it's likely very easy to adapt yours to report results back to Test Swarm.
This is great but sits outside our continuous integration environment which runs our server side tests whenever someone pushes code. Ideally we want someway to trigger Test Swarm in parallel to our server side tests.
How I've done it
- Firstly you'll need to install my fork of Test Swarm (http://github.com/carlmw/testswarm) which includes a couple of scripts to generate xUnit test reports and feed them into Hudson. Instructions can be found in the README.
- Hudson, If you're using something other than Hudson I'm sure that this technique can be adapted for your environment (This should all be identical for Jenkins though I won't know for sure until we migrate to it) .
- Hudson xUnit plugin.
Tying Test Swarm into Hudson
Firstly you'll need to make sure your tests are available to all your Test Swarm clients when the code for your build is checked out. Publishing the directory in your build workspace containing your tests through apache would likely be the easiest way to achieve this.
Add a job to Test Swarm that runs your tests against your desired browser set. Make a note of the jobs ID from the address bar.
Clearing current test state
In the configure page for your Hudson build, add/modify a shell script build step and before the command that runs your server side tests but after the one that checks out the code for your build — Add the following cURL command.
curl --data "job_id=%JOB_ID%&state=wipejob&type=reset" http://swarm-server
Replace %JOB_ID% with the id of your Test Swarm job and change http://swarm-server to point to your own Test Swarm server.
That cURL command is important and will reset the results of the job prior to running your tests.
Capturing test results
Capturing the results of your tests and feeding them back into Hudson is pretty easy.
Add the following command after your server side test runner. Doing it this way ensures that your server and front end tests can run in parallel.
php /path/to/testswarm/scripts/get_xunit_results.php http://swarm-server/job/%JOB_ID%/ /results/path/
Replace /path/to/testswarm/scripts/get_xunit_results.php with the correct path to that script on your Hudson server, also replace http://swarm-server/job/%JOB_ID%/ with the URL of the Test Swarm job you want to capture results for. Finally replace /results/xml/ with the path to the directory in which you want to write the xUnit test results (this directory will be emptied automatically before the results are written).
Now that you have your test results you'll need to tell Hudson where to find them; add the path (E.g. /path/to/results/*.xml) to your Test Swarm xUnit test results to the xUnit config in your build.
If all has gone well you should be able to run your build and results from Test Swarm will be fed into your Hudson build.
Next up i'll be generating code coverage reports and feeding them into Hudson!