Difference between revisions of "Jenkins"

Line 4: Line 4:
 
=The CMS Jenkins Server=  
 
=The CMS Jenkins Server=  
  
[http://en.wikipedia.org/wiki/Continuous_integration | Continuous integration] is a software development practice where every time code is committed to version control tests get automatically run to ensure that new changes haven't broken any functionality. This is especially useful when a large team is working on the same piece of software to ensure that changes made by different groups don't conflict with each other. It's also useful for solo developers to have tests run automatically as you don't have to remember to continuously run tests manually after every change.
+
[http://en.wikipedia.org/wiki/Continuous_integration Continuous integration] is a software development practice where every time code is committed to version control tests get automatically run to ensure that new changes haven't broken any functionality. This is especially useful when a large team is working on the same piece of software to ensure that changes made by different groups don't conflict with each other. It's also useful for solo developers to have tests run automatically as you don't have to remember to continuously run tests manually after every change.
  
The CMS team has a Jenkins server at https://climate-cms.nci.org.au/jenkins that you can use to test your model code. Jenkins is an open-source continuous integration server that monitors version controlled repositories & then runs set commands when code changes are made. Our Jenkins server is set up so that it builds and runs your code on Raijin, so you have all the resources of the HPC including the PBS queue and the NCI software modules.
+
The CMS team has a Jenkins server at https://accessdev.nci.org.au/jenkins that you can use to test your model code. Jenkins is an open-source continuous integration server that monitors version controlled repositories & then runs set commands when code changes are made. Our Jenkins server is set up so that it builds and runs your code on Raijin, so you have all the resources of the HPC including the PBS queue and the NCI software modules.
  
 
=Registering to use Jenkins=  
 
=Registering to use Jenkins=  
Line 24: Line 24:
 
==Alternatives to Jenkins==  
 
==Alternatives to Jenkins==  
  
Depending on your needs you may not need to run your tests on Raijin, for instance if you don't need access to `ifort` or any of the RDSI data. In this case you may want to look at services like [https://travis-ci.org | Travis], which is a free CI server for open-source code. Travis runs your code on a virtual Ubuntu machine where you can install whatever dependencies you need and links very easily to Github. Consider making use of this whenever you can, as it means you don't have to use your NCI resources for tests!
+
Depending on your needs you may not need to run your tests on Raijin, for instance if you don't need access to `ifort` or any of the RDSI data. In this case you may want to look at services like [https://travis-ci.org Travis], which is a free CI server for open-source code. Travis runs your code on a virtual Ubuntu machine where you can install whatever dependencies you need and links very easily to Github. Consider making use of this whenever you can, as it means you don't have to use your NCI resources for tests!
  
 
= Writing Tests =
 
= Writing Tests =
Line 34: Line 34:
 
Unit tests aim to test small individual 'units' of code, for instance a single function. They run quickly, as each individual test is very simple, so it's easy to run them as you're coding. Generally it's easiest to add them in at the start of a project using test-driven development, it's harder to come back after the fact and add lots of tests. Since each test is simple it's easy to locate failures, as well as making sure that all code paths are taken. Having lots of small tests can make it hard to make sweeping changes to your code, as lots of tests need to change with it, and it can be difficult to test the overlap between different units.
 
Unit tests aim to test small individual 'units' of code, for instance a single function. They run quickly, as each individual test is very simple, so it's easy to run them as you're coding. Generally it's easiest to add them in at the start of a project using test-driven development, it's harder to come back after the fact and add lots of tests. Since each test is simple it's easy to locate failures, as well as making sure that all code paths are taken. Having lots of small tests can make it hard to make sweeping changes to your code, as lots of tests need to change with it, and it can be difficult to test the overlap between different units.
  
There are many different unit testing libraries available for different languages. [http://sourceforge.net/projects/pfunit/ | pFUnit] is a good starting point for testing MPI Fortran programs, [https://nose.readthedocs.org/en/latest/ | nose] is a popular framework for Python
+
There are many different unit testing libraries available for different languages. [http://sourceforge.net/projects/pfunit/ pFUnit] is a good starting point for testing MPI Fortran programs, [https://nose.readthedocs.org/en/latest/ nose] is a popular framework for Python
  
 
== Integration Tests ==
 
== Integration Tests ==
  
 
Integration tests check how your code works as a complete package, doing operations like a user would, though perhaps on simpler data. For instance if you're developing a model you might run the model at low resolution for a day, comparing the model output with a previous result to see if it has changed. Integration tests are simple to create if you're just doing a 'diff' on the output, but take longer to run and make it harder to locate the source of errors compared to unit tests.
 
Integration tests check how your code works as a complete package, doing operations like a user would, though perhaps on simpler data. For instance if you're developing a model you might run the model at low resolution for a day, comparing the model output with a previous result to see if it has changed. Integration tests are simple to create if you're just doing a 'diff' on the output, but take longer to run and make it harder to locate the source of errors compared to unit tests.
 
=Creating an Automatic Build Job=
 
 
==Getting notified when tests fail==
 
 
==Linking with a Github project==
 

Revision as of 23:47, 27 November 2019

Template:Stub This is a stub page and needs expansion

The CMS Jenkins Server

Continuous integration is a software development practice where every time code is committed to version control tests get automatically run to ensure that new changes haven't broken any functionality. This is especially useful when a large team is working on the same piece of software to ensure that changes made by different groups don't conflict with each other. It's also useful for solo developers to have tests run automatically as you don't have to remember to continuously run tests manually after every change.

The CMS team has a Jenkins server at https://accessdev.nci.org.au/jenkins that you can use to test your model code. Jenkins is an open-source continuous integration server that monitors version controlled repositories & then runs set commands when code changes are made. Our Jenkins server is set up so that it builds and runs your code on Raijin, so you have all the resources of the HPC including the PBS queue and the NCI software modules.

Registering to use Jenkins

There is a little bit of setup to add new users to Jenkins, so we ask that you let the CMS team know if you'd like to make use of it. Jenkins connects to Raijin using your own account with a SSH key, you'll be given a public key to add to your `~/.ssh/authorized_keys` file on Raijin.

Account Security

In order for Jenkins to be able to build your code it needs to be able to connect as you to Raijin. This means that it will use your own NCI resources, including compute time and storage, and any actions Jenkins takes (e.g. if you ask Jenkins to delete files as part of a build) will happen as if you did it yourself on Raijin.

The SSH key Jenkins uses to connect to Raijin uses a secure passphrase, but by necessity Jenkins has to store this in memory to be able to automatically run builds. It is important that the public key in your `~/.ssh/authorized_keys` file is secured like this:

from="130.56.244.115",no-port-forwarding,no-x11-forwarding,no-agent-forwarding ssh-rsa xxxxxxxxxxxxxxxxxxx jenkins@climate-cms.nci.org.au

This means that the key can only be used from the Jenkins server (which has the IP address 130.56.244.115) and that connections can't be forwarded to other machines.

Alternatives to Jenkins

Depending on your needs you may not need to run your tests on Raijin, for instance if you don't need access to `ifort` or any of the RDSI data. In this case you may want to look at services like Travis, which is a free CI server for open-source code. Travis runs your code on a virtual Ubuntu machine where you can install whatever dependencies you need and links very easily to Github. Consider making use of this whenever you can, as it means you don't have to use your NCI resources for tests!

Writing Tests

Before you add your code to Jenkins you should first make some tests to run. There are a few different testing philosophies, each with their own pros and cons. Much has been written about testing on the Internet, here are some brief summaries:

Unit Tests

Unit tests aim to test small individual 'units' of code, for instance a single function. They run quickly, as each individual test is very simple, so it's easy to run them as you're coding. Generally it's easiest to add them in at the start of a project using test-driven development, it's harder to come back after the fact and add lots of tests. Since each test is simple it's easy to locate failures, as well as making sure that all code paths are taken. Having lots of small tests can make it hard to make sweeping changes to your code, as lots of tests need to change with it, and it can be difficult to test the overlap between different units.

There are many different unit testing libraries available for different languages. pFUnit is a good starting point for testing MPI Fortran programs, nose is a popular framework for Python

Integration Tests

Integration tests check how your code works as a complete package, doing operations like a user would, though perhaps on simpler data. For instance if you're developing a model you might run the model at low resolution for a day, comparing the model output with a previous result to see if it has changed. Integration tests are simple to create if you're just doing a 'diff' on the output, but take longer to run and make it harder to locate the source of errors compared to unit tests.