Difference between revisions of "ACCESS-ESM-CSIRO"

(Created page with "This page mainly documents some experiences with the ACCESS-ESM model run with the CSIRO scripts. We strongly recommend using the payu version instead. There is some document...")
 
 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
[[Category:ACCESS 1.5]]
 +
 
This page mainly documents some experiences with the ACCESS-ESM model run with the CSIRO scripts.
 
This page mainly documents some experiences with the ACCESS-ESM model run with the CSIRO scripts.
We strongly recommend using the payu version instead.
+
We strongly recommend using the [[ACCESS-ESM|payu version]] instead.
  
 
There is some documentation on google drive, CMS > Coupled Models
 
There is some documentation on google drive, CMS > Coupled Models
  
 
+
<syntaxhighlight lang="bash">
<syntaxhighlight lang="unknown">
 
 
git clone git@bitbucket.org:climate-cms/csiro-scripts.git
 
git clone git@bitbucket.org:climate-cms/csiro-scripts.git
 
cd csiro-scripts/original/build
 
cd csiro-scripts/original/build
Line 22: Line 23:
  
 
&nbsp;
 
&nbsp;
<syntaxhighlight lang="unknown">
+
<syntaxhighlight lang="bash">
 
rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29*
 
rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29*
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 29: Line 30:
  
 
&nbsp;
 
&nbsp;
<syntaxhighlight lang="unknown">
+
<syntaxhighlight lang="bash">
 
#PBS -P w35
 
#PBS -P w35
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 36: Line 37:
  
 
&nbsp;
 
&nbsp;
<syntaxhighlight lang="unknown">
+
<syntaxhighlight lang="bash">
 
#-- Initial and Final Date of the Experiment
 
#-- Initial and Final Date of the Experiment
 
iniyear=1;      finalyear=1;            typeset -Z4 iniyear  finalyear
 
iniyear=1;      finalyear=1;            typeset -Z4 iniyear  finalyear
Line 46: Line 47:
  
 
&nbsp;
 
&nbsp;
<syntaxhighlight lang="unknown">
+
<syntaxhighlight lang="bash">
 
qsub test1
 
qsub test1
 
</syntaxhighlight>
 
</syntaxhighlight>

Latest revision as of 00:08, 12 December 2019


This page mainly documents some experiences with the ACCESS-ESM model run with the CSIRO scripts. We strongly recommend using the payu version instead.

There is some documentation on google drive, CMS > Coupled Models

git clone git@bitbucket.org:climate-cms/csiro-scripts.git
cd csiro-scripts/original/build
make

Turns out I needed access to the CABLE repo, see https://trac.nci.org.au/trac/cable/wiki/CableRegistration

Other than that, compile worked fine.

Running first test

copy the scripts PI-C2C-1p5r29* from the original directory to a new experiment directory.

Rename the scripts with

 

rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29*

Make sure that the correct project is set in test1:

 

#PBS -P w35

Change initial date and final date in test1.init:

 

#-- Initial and Final Date of the Experiment
iniyear=1;      finalyear=1;            typeset -Z4 iniyear  finalyear
inimonth=1;     finalmonth=1;           typeset -Z2 inimonth finalmonth
iniday=1;       finalday=5;             typeset -Z2 iniday   finalday

Then run

 

qsub test1

Link the build directory to the experiment directory.

Issues

The RUNID of PI-C2C-1p5r29 was hardcoded in the script. I removed that hardcoded line and left the export RUNID=`basename $PBS_O_WORKDIR` in because the experiment directory has the same name as the scripts.

It didn't find the build directory in the experiment directory. I linked the build directory there.

The model failed asking for a restart dump that didn't exist. Deleting the test1.date file solved that.

Didn't find create_rankfile.py -- copied from original directory. (Had to delete test1.date again.)

I also pre-emptively linked the runscripts directory from original to my experiment directory.

How ACCESS works

<RUNID>

The actual sumit script does not have an extension, and is a ksh script. It sets a whole lot of environment variables, most of them exported.

It then sources, if present, runscripts/umprofile which also sets some variables, however, many of these variables don't seem to refer to a path that exists on raijin. Then, again, if present, runs runscripts/setglobalvars, again, the contents of which seem to be outdated.

It sets a few more variables, then sources <RUNID>.init.

Many more variable declarations, then it sources UMScr_Toplevel.

Finally, it sources <RUNID>.fin

<RUNID>.init

This script is *sourced* by <RUNID>, so it's still a ksh script. It begins with a function declaration, which sets certain ancillary files depending on the year.

Then come many more variables, including:

  • CMIP5RUN, which can be any of these: picontrolv,historical,pi4xCO2,pi1pcntCO2,rcp45,rcp85,rcp26.
  • nproc_ice (12 currently)
  • oce_nx and oce_ny for ocean decomposition (currently 12 and 4, respectively)
  • ntproc as the total number of cores for this job (UM_NPES + nproc_ice + (oce_nx * oce_ny))
  • iniyear, inimonth, and iniday as the initial date of the model, which is then compressed into inidate (YYYYMMDD)
  • finalyear, finalmonth, finalday as the final date of the model, compressed into finaldate.

Then it looks for <RUNID>.date:

If it doesn't exist, then it assumes that it's a new run. It creates the file and dumps the initial dates in there. If the file does exist, it reads the date out of it (only last line).

It then uses ~access/bin/calendar_more to calculate the times for this run (initial date, end date, first date of next run, et cetera).

From this, it gets, amongst other things, the days_in_run, which it then multiplies by 86,400 (secs per day) to get runtime.

Next, the script creates it work and run directories, copying files there as needed. (Apparently, for historical runs, it also needs to change the dates in the UM files, changing the reference date from the 16th of the month to the first.) This is also where it uses the function it declared at the beginning.

Finally, it makes several (currently obscure) changes to a lot of namelists, and then creates the ACCESSRUNCMD.

And it loads new modules, replacing others if they had been loaded before.

<RUNID>.fin

Check whether variable FCODE is 0 (presumably the return code of the Model run?), only then will it do anything.

It calculates the name of the restart file restartfile="aiihca.da${umdate}" where umdate has been created by datetoum or datetoum2.

Next, it moves what I think are the coupling restart files, ${cplrundir}/?2?.nc to the archive ${archivedir}/restart/cpl/$resfile-${enddate}

Then, it accesses $atmrundir: Validates the date of the restart file, moves it to ${archivedir}/restart/atm/${restartarch} ((restartarch had been set to "${RUNID}.astart-${nextdate}").

Next, it moves all UM output files (aiihca.p*) to the archive dir, renaming them in the process.

Modifications

Created a new branch: holger_testing

MOM5 Version Control

MOM5 is under version control: https://github.com/OceansAus/ACCESS-ESM1.5-MOM5.git - this was added to the Makefile.

ummodel_hg3

ummodel_hg3 was still copied from /short/p66/txz599/ACCESSHOME/submodels/UM/ummodel_hg3/. That directory itself is under version control, but with some non-checked-in.

umbase_hg3

umbase_hg3 was still copied. But there weren't any interesting changes in the directory compared the the repository. (Only fcm_env.sh and parsed_bld.cfg) So I've changed it to point to the svn repo.

bld-hadgem3-mct.cfg

The file bld-hadgem3-mct.cfg was missing from the ummodel_hg3 repo. For now, I've made it part of the access-esm repo, and added a line in the Makefile to copy it over.

Additional information

CSIRO documentation: https://access-svn.nci.org.au/trac/cmip5/wiki