Difference between revisions of "Payuongadi"

(Instructions for transitioning a payu experiment to gadi)
 
Line 10: Line 10:
 
<span class="marker">LINK TO NEW CONDA LOAD INSTRUCTIONS</span>
 
<span class="marker">LINK TO NEW CONDA LOAD INSTRUCTIONS</span>
  
There is not short file system&nbsp;
+
There is no&nbsp;short file system on gadi. This has been replaced by a time-limited scratch file space. The latest version of payu has changed the default location for payu laboratories to <code>/scratch/$PROJECT/$USER/$MODEL</code>.
 +
 
 +
However, if your <code>config.yaml</code> hard codes <code>shortpath</code> to a location under <code>/short</code> this will have to be changed. This is a common practice so that a model can be submitted to the PBS system under more than one project code, but ensure that all files are read to and from the same laboratory space.
 +
 
 +
For example, if you <code>config.yaml</code> contains the following;
 +
 
 +
<code>shortpath: /short/v45</code>
 +
 
 +
you will need to change it. Initially the recommendation would be to change it to this:
 +
 
 +
<code>shortpath: /scratch/v45</code>
 +
 
 +
but this comes with some drawbacks. The <code>/scratch</code> disk will be purged based on the modification time of files, so some care must be taken to ensure files which must be kept are transferred to other filesystems before they are deleted.

Revision as of 00:50, 18 October 2019

payu on gadi

 

Using payu on the new gadi HPC system should be straightforward. Depending on the setup of your configuration you might need to make very few changes.

Everyone will need to load the CMS conda module conda/analysis3-unstable from it's new location. Instructions are here:

LINK TO NEW CONDA LOAD INSTRUCTIONS

There is no short file system on gadi. This has been replaced by a time-limited scratch file space. The latest version of payu has changed the default location for payu laboratories to /scratch/$PROJECT/$USER/$MODEL.

However, if your config.yaml hard codes shortpath to a location under /short this will have to be changed. This is a common practice so that a model can be submitted to the PBS system under more than one project code, but ensure that all files are read to and from the same laboratory space.

For example, if you config.yaml contains the following;

shortpath: /short/v45

you will need to change it. Initially the recommendation would be to change it to this:

shortpath: /scratch/v45

but this comes with some drawbacks. The /scratch disk will be purged based on the modification time of files, so some care must be taken to ensure files which must be kept are transferred to other filesystems before they are deleted.