The CMS Jenkins Server
Continuous integration is a software development practice where every time code is committed to version control tests get automatically run to ensure that new changes haven't broken any functionality. This is especially useful when a large team is working on the same piece of software to ensure that changes made by different groups don't conflict with each other. It's also useful for solo developers to have tests run automatically as you don't have to remember to continuously run tests manually after every change.
The CMS team has a Jenkins server at https://accessdev.nci.org.au/jenkins that you can use to test your model code. Jenkins is an open-source continuous integration server that monitors version controlled repositories & then runs set commands when code changes are made. Our Jenkins server is set up so that it builds and runs your code on Raijin, so you have all the resources of the HPC including the PBS queue and the NCI software modules.
Registering to use Jenkins
There is a little bit of setup to add new users to Jenkins, so we ask that you let the CMS team know if you'd like to make use of it. Jenkins connects to Raijin using your own account with a SSH key, you'll be given a public key to add to your `~/.ssh/authorized_keys` file on Raijin.
In order for Jenkins to be able to build your code it needs to be able to connect as you to Raijin. This means that it will use your own NCI resources, including compute time and storage, and any actions Jenkins takes (e.g. if you ask Jenkins to delete files as part of a build) will happen as if you did it yourself on Raijin.
The SSH key Jenkins uses to connect to Raijin uses a secure passphrase, but by necessity Jenkins has to store this in memory to be able to automatically run builds. It is important that the public key in your `~/.ssh/authorized_keys` file is secured like this:
from="18.104.22.168",no-port-forwarding,no-x11-forwarding,no-agent-forwarding ssh-rsa xxxxxxxxxxxxxxxxxxx firstname.lastname@example.org
This means that the key can only be used from the Jenkins server (which has the IP address 22.214.171.124) and that connections can't be forwarded to other machines.
Alternatives to Jenkins
Depending on your needs you may not need to run your tests on Raijin, for instance if you don't need access to `ifort` or any of the RDSI data. In this case you may want to look at services like Travis, which is a free CI server for open-source code. Travis runs your code on a virtual Ubuntu machine where you can install whatever dependencies you need and links very easily to Github. Consider making use of this whenever you can, as it means you don't have to use your NCI resources for tests!
Before you add your code to Jenkins you should first make some tests to run. There are a few different testing philosophies, each with their own pros and cons. Much has been written about testing on the Internet, here are some brief summaries:
Unit tests aim to test small individual 'units' of code, for instance a single function. They run quickly, as each individual test is very simple, so it's easy to run them as you're coding. Generally it's easiest to add them in at the start of a project using test-driven development, it's harder to come back after the fact and add lots of tests. Since each test is simple it's easy to locate failures, as well as making sure that all code paths are taken. Having lots of small tests can make it hard to make sweeping changes to your code, as lots of tests need to change with it, and it can be difficult to test the overlap between different units.
Integration tests check how your code works as a complete package, doing operations like a user would, though perhaps on simpler data. For instance if you're developing a model you might run the model at low resolution for a day, comparing the model output with a previous result to see if it has changed. Integration tests are simple to create if you're just doing a 'diff' on the output, but take longer to run and make it harder to locate the source of errors compared to unit tests.