Difference between revisions of "ACCESS-ESM 1.5"
m |
|||
Line 1: | Line 1: | ||
+ | |||
{{Stub}} | {{Stub}} | ||
− | Payu configuration: https://github.com/coecms/esm-pre-industrial | + | Payu configuration: [https://github.com/coecms/esm-pre-industrial https://github.com/coecms/esm-pre-industrial] |
See the README on the github page for information on how to run ACCESS-ESM with Payu | See the README on the github page for information on how to run ACCESS-ESM with Payu | ||
− | CSIRO documentation: https://access-svn.nci.org.au/trac/cmip5/wiki | + | CSIRO documentation: [https://access-svn.nci.org.au/trac/cmip5/wiki https://access-svn.nci.org.au/trac/cmip5/wiki] |
− | |||
+ | | ||
= Payu Experiments = | = Payu Experiments = | ||
Line 24: | Line 25: | ||
=== Last Interglacial === | === Last Interglacial === | ||
+ | |||
+ | Main Page: [[ACCESS-ESM-LIG|ACCESS-ESM-LIG]] | ||
Science Contacts: Nick Yeung, Laurie Menviel | Science Contacts: Nick Yeung, Laurie Menviel | ||
Line 31: | Line 34: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | =CSIRO KSH Scripts= | + | = CSIRO KSH Scripts = |
There is some documentation on google drive, CMS > Coupled Models | There is some documentation on google drive, CMS > Coupled Models | ||
− | <syntaxhighlight> | + | |
+ | <syntaxhighlight lang="unknown"> | ||
git clone git@bitbucket.org:climate-cms/csiro-scripts.git | git clone git@bitbucket.org:climate-cms/csiro-scripts.git | ||
cd csiro-scripts/original/build | cd csiro-scripts/original/build | ||
Line 41: | Line 45: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | Turns out I needed access to the CABLE repo, see https://trac.nci.org.au/trac/cable/wiki/CableRegistration | + | Turns out I needed access to the CABLE repo, see [https://trac.nci.org.au/trac/cable/wiki/CableRegistration https://trac.nci.org.au/trac/cable/wiki/CableRegistration] |
Other than that, compile worked fine. | Other than that, compile worked fine. | ||
− | ===Running first test=== | + | === Running first test === |
copy the scripts <code>PI-C2C-1p5r29*</code> from the <code>original</code> directory to a new experiment directory. | copy the scripts <code>PI-C2C-1p5r29*</code> from the <code>original</code> directory to a new experiment directory. | ||
Line 51: | Line 55: | ||
Rename the scripts with | Rename the scripts with | ||
− | <syntaxhighlight> | + | |
+ | <syntaxhighlight lang="unknown"> | ||
rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29* | rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29* | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 57: | Line 62: | ||
Make sure that the correct project is set in <code>test1</code>: | Make sure that the correct project is set in <code>test1</code>: | ||
− | <syntaxhighlight> | + | |
+ | <syntaxhighlight lang="unknown"> | ||
#PBS -P w35 | #PBS -P w35 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 63: | Line 69: | ||
Change initial date and final date in <code>test1.init</code>: | Change initial date and final date in <code>test1.init</code>: | ||
− | <syntaxhighlight> | + | |
+ | <syntaxhighlight lang="unknown"> | ||
#-- Initial and Final Date of the Experiment | #-- Initial and Final Date of the Experiment | ||
iniyear=1; finalyear=1; typeset -Z4 iniyear finalyear | iniyear=1; finalyear=1; typeset -Z4 iniyear finalyear | ||
Line 72: | Line 79: | ||
Then run | Then run | ||
− | <syntaxhighlight> | + | |
+ | <syntaxhighlight lang="unknown"> | ||
qsub test1 | qsub test1 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 78: | Line 86: | ||
Link the <code>build</code> directory to the experiment directory. | Link the <code>build</code> directory to the experiment directory. | ||
− | ====Issues==== | + | ==== Issues ==== |
The RUNID of <code>PI-C2C-1p5r29</code> was hardcoded in the script. I removed that hardcoded line and left the <code>export RUNID=`basename $PBS_O_WORKDIR`</code> in because the experiment directory has the same name as the scripts. | The RUNID of <code>PI-C2C-1p5r29</code> was hardcoded in the script. I removed that hardcoded line and left the <code>export RUNID=`basename $PBS_O_WORKDIR`</code> in because the experiment directory has the same name as the scripts. | ||
Line 90: | Line 98: | ||
I also pre-emptively linked the <code>runscripts</code> directory from <code>original</code> to my experiment directory. | I also pre-emptively linked the <code>runscripts</code> directory from <code>original</code> to my experiment directory. | ||
− | ==How ACCESS works== | + | == How ACCESS works == |
− | ===<RUNID>=== | + | === <RUNID> === |
− | The actual sumit script does not have an extension, and is a <code>ksh</code> script. | + | The actual sumit script does not have an extension, and is a <code>ksh</code> script. It sets a whole lot of environment variables, most of them exported. |
− | It sets a whole lot of environment variables, most of them exported. | ||
− | It then sources, if present, <code>runscripts/umprofile</code> which also sets some variables, however, many of these variables don't seem to refer to a path that exists on <code>raijin</code>. | + | It then sources, if present, <code>runscripts/umprofile</code> which also sets some variables, however, many of these variables don't seem to refer to a path that exists on <code>raijin</code>. Then, again, if present, runs <code>runscripts/setglobalvars</code>, again, the contents of which seem to be outdated. |
− | Then, again, if present, runs <code>runscripts/setglobalvars</code>, again, the contents of which seem to be outdated. | ||
It sets a few more variables, then sources <code><RUNID>.init</code>. | It sets a few more variables, then sources <code><RUNID>.init</code>. | ||
Line 106: | Line 112: | ||
Finally, it sources <code><RUNID>.fin</code> | Finally, it sources <code><RUNID>.fin</code> | ||
− | ===<RUNID>.init=== | + | === <RUNID>.init === |
− | This script is *sourced* by <code><RUNID></code>, so it's still a ksh script. | + | This script is *sourced* by <code><RUNID></code>, so it's still a ksh script. It begins with a function declaration, which sets certain ancillary files depending on the year. |
− | It begins with a function declaration, which sets certain ancillary files depending on the year. | ||
Then come many more variables, including: | Then come many more variables, including: | ||
− | * <code>CMIP5RUN</code>, which can be any of these: <code>picontrolv,historical,pi4xCO2,pi1pcntCO2,rcp45,rcp85,rcp26</code>. | + | *<code>CMIP5RUN</code>, which can be any of these: <code>picontrolv,historical,pi4xCO2,pi1pcntCO2,rcp45,rcp85,rcp26</code>. |
− | * <code>nproc_ice</code> (12 currently) | + | *<code>nproc_ice</code> (12 currently) |
− | * <code>oce_nx</code> and <code>oce_ny</code> for ocean decomposition (currently 12 and 4, respectively) | + | *<code>oce_nx</code> and <code>oce_ny</code> for ocean decomposition (currently 12 and 4, respectively) |
− | * <code>ntproc</code> as the total number of cores for this job (<code>UM_NPES</code> + <code>nproc_ice</code> + <code>(oce_nx * oce_ny)</code>) | + | *<code>ntproc</code> as the total number of cores for this job (<code>UM_NPES</code> + <code>nproc_ice</code> + <code>(oce_nx * oce_ny)</code>) |
− | * <code>iniyear</code>, <code>inimonth</code>, and <code>iniday</code> as the initial date of the model, which is then compressed into <code>inidate</code> (<code>YYYYMMDD</code>) | + | *<code>iniyear</code>, <code>inimonth</code>, and <code>iniday</code> as the initial date of the model, which is then compressed into <code>inidate</code> (<code>YYYYMMDD</code>) |
− | * <code>finalyear</code>, <code>finalmonth</code>, <code>finalday</code> as the final date of the model, compressed into <code>finaldate</code>. | + | *<code>finalyear</code>, <code>finalmonth</code>, <code>finalday</code> as the final date of the model, compressed into <code>finaldate</code>. |
Then it looks for <code><RUNID>.date</code>: | Then it looks for <code><RUNID>.date</code>: | ||
− | If it '''doesn't''' exist, then it assumes that it's a new run. It creates the file and dumps the initial dates in there. | + | If it '''doesn't''' exist, then it assumes that it's a new run. It creates the file and dumps the initial dates in there. If the file '''does''' exist, it reads the date out of it (only last line). |
− | If the file '''does''' exist, it reads the date out of it (only last line). | ||
It then uses <code>~access/bin/calendar_more</code> to calculate the times for this run (initial date, end date, first date of next run, et cetera). | It then uses <code>~access/bin/calendar_more</code> to calculate the times for this run (initial date, end date, first date of next run, et cetera). | ||
Line 129: | Line 133: | ||
From this, it gets, amongst other things, the <code>days_in_run</code>, which it then multiplies by 86,400 (secs per day) to get <code>runtime</code>. | From this, it gets, amongst other things, the <code>days_in_run</code>, which it then multiplies by 86,400 (secs per day) to get <code>runtime</code>. | ||
− | Next, the script creates it work and run directories, copying files there as needed. (Apparently, for historical runs, it also needs to change the dates in the UM files, changing the reference date from the 16th of the month to the first.) | + | Next, the script creates it work and run directories, copying files there as needed. (Apparently, for historical runs, it also needs to change the dates in the UM files, changing the reference date from the 16th of the month to the first.) This is also where it uses the function it declared at the beginning. |
− | This is also where it uses the function it declared at the beginning. | ||
Finally, it makes several (currently obscure) changes to a lot of namelists, and then creates the <code>ACCESSRUNCMD</code>. | Finally, it makes several (currently obscure) changes to a lot of namelists, and then creates the <code>ACCESSRUNCMD</code>. | ||
Line 136: | Line 139: | ||
And it loads new modules, replacing others if they had been loaded before. | And it loads new modules, replacing others if they had been loaded before. | ||
− | ===<RUNID>.fin=== | + | === <RUNID>.fin === |
Check whether variable <code>FCODE</code> is 0 (presumably the return code of the Model run?), only then will it do anything. | Check whether variable <code>FCODE</code> is 0 (presumably the return code of the Model run?), only then will it do anything. | ||
Line 148: | Line 151: | ||
Next, it moves all UM output files (<code>aiihca.p*</code>) to the archive dir, renaming them in the process. | Next, it moves all UM output files (<code>aiihca.p*</code>) to the archive dir, renaming them in the process. | ||
− | ==Modifications== | + | == Modifications == |
Created a new branch: <code>holger_testing</code> | Created a new branch: <code>holger_testing</code> | ||
− | ===MOM5 Version Control=== | + | === MOM5 Version Control === |
− | MOM5 is under version control: <code>https://github.com/OceansAus/ACCESS-ESM1.5-MOM5.git</code> - this was added to the <code>Makefile</code>. | + | MOM5 is under version control: <code>[https://github.com/OceansAus/ACCESS-ESM1.5-MOM5.git https://github.com/OceansAus/ACCESS-ESM1.5-MOM5.git]</code> - this was added to the <code>Makefile</code>. |
− | ===ummodel_hg3=== | + | === ummodel_hg3 === |
<code>ummodel_hg3</code> was still copied from <code>/short/p66/txz599/ACCESSHOME/submodels/UM/ummodel_hg3/</code>. That directory itself is under version control, but with some non-checked-in. | <code>ummodel_hg3</code> was still copied from <code>/short/p66/txz599/ACCESSHOME/submodels/UM/ummodel_hg3/</code>. That directory itself is under version control, but with some non-checked-in. | ||
− | ===umbase_hg3=== | + | === umbase_hg3 === |
− | <code>umbase_hg3</code> was still copied. But there weren't any interesting changes in the directory compared the the repository. (Only <code>fcm_env.sh</code> and <code>parsed_bld.cfg</code>) | + | <code>umbase_hg3</code> was still copied. But there weren't any interesting changes in the directory compared the the repository. (Only <code>fcm_env.sh</code> and <code>parsed_bld.cfg</code>) So I've changed it to point to the svn repo. |
− | So I've changed it to point to the svn repo. | ||
− | ===bld-hadgem3-mct.cfg=== | + | === bld-hadgem3-mct.cfg === |
The file <code>bld-hadgem3-mct.cfg</code> was missing from the <code>ummodel_hg3</code> repo. For now, I've made it part of the access-esm repo, and added a line in the <code>Makefile</code> to copy it over. | The file <code>bld-hadgem3-mct.cfg</code> was missing from the <code>ummodel_hg3</code> repo. For now, I've made it part of the access-esm repo, and added a line in the <code>Makefile</code> to copy it over. | ||
− | + | [[Category:ACCESS 1.5]] [[Category:Coupled Models]] | |
− | [[Category: ACCESS 1.5]][[Category: Coupled Models]] |
Revision as of 00:03, 15 July 2019
Payu configuration: https://github.com/coecms/esm-pre-industrial
See the README on the github page for information on how to run ACCESS-ESM with Payu
CSIRO documentation: https://access-svn.nci.org.au/trac/cmip5/wiki
Contents
Payu Experiments
We strongly recommend that you run ACCESS-ESM 1.5 using the payu driver. You can download the configuration by typing
$ git clone https://github.com/coecms/esm-pre-industrial.git
The website (https://github.com/coecms/esm-pre-industrial) of the configuration also contains detailed instructions on how to run the model.
Configurations
piControl
Default configuration, based off CSIRO ksh scripts
Last Interglacial
Main Page: ACCESS-ESM-LIG
Science Contacts: Nick Yeung, Laurie Menviel
Note: In addition to changed MMRs for certain gases, it also needs an updated UM build with different orbital parameters. This updated UM build is at
/short/public/access-esm/payu/bin/coe/um_hg3.exe-20190412-pmip-r327
CSIRO KSH Scripts
There is some documentation on google drive, CMS > Coupled Models
git clone git@bitbucket.org:climate-cms/csiro-scripts.git
cd csiro-scripts/original/build
make
Turns out I needed access to the CABLE repo, see https://trac.nci.org.au/trac/cable/wiki/CableRegistration
Other than that, compile worked fine.
Running first test
copy the scripts PI-C2C-1p5r29*
from the original
directory to a new experiment directory.
Rename the scripts with
rename PI-C2C-1p5r29 test1 PI-C2C-1p5r29*
Make sure that the correct project is set in test1
:
#PBS -P w35
Change initial date and final date in test1.init
:
#-- Initial and Final Date of the Experiment
iniyear=1; finalyear=1; typeset -Z4 iniyear finalyear
inimonth=1; finalmonth=1; typeset -Z2 inimonth finalmonth
iniday=1; finalday=5; typeset -Z2 iniday finalday
Then run
qsub test1
Link the build
directory to the experiment directory.
Issues
The RUNID of PI-C2C-1p5r29
was hardcoded in the script. I removed that hardcoded line and left the export RUNID=`basename $PBS_O_WORKDIR`
in because the experiment directory has the same name as the scripts.
It didn't find the build directory in the experiment directory. I linked the build directory there.
The model failed asking for a restart dump that didn't exist. Deleting the test1.date
file solved that.
Didn't find create_rankfile.py
-- copied from original
directory. (Had to delete test1.date
again.)
I also pre-emptively linked the runscripts
directory from original
to my experiment directory.
How ACCESS works
<RUNID>
The actual sumit script does not have an extension, and is a ksh
script. It sets a whole lot of environment variables, most of them exported.
It then sources, if present, runscripts/umprofile
which also sets some variables, however, many of these variables don't seem to refer to a path that exists on raijin
. Then, again, if present, runs runscripts/setglobalvars
, again, the contents of which seem to be outdated.
It sets a few more variables, then sources <RUNID>.init
.
Many more variable declarations, then it sources UMScr_Toplevel
.
Finally, it sources <RUNID>.fin
<RUNID>.init
This script is *sourced* by <RUNID>
, so it's still a ksh script. It begins with a function declaration, which sets certain ancillary files depending on the year.
Then come many more variables, including:
CMIP5RUN
, which can be any of these:picontrolv,historical,pi4xCO2,pi1pcntCO2,rcp45,rcp85,rcp26
.nproc_ice
(12 currently)oce_nx
andoce_ny
for ocean decomposition (currently 12 and 4, respectively)ntproc
as the total number of cores for this job (UM_NPES
+nproc_ice
+(oce_nx * oce_ny)
)iniyear
,inimonth
, andiniday
as the initial date of the model, which is then compressed intoinidate
(YYYYMMDD
)finalyear
,finalmonth
,finalday
as the final date of the model, compressed intofinaldate
.
Then it looks for <RUNID>.date
:
If it doesn't exist, then it assumes that it's a new run. It creates the file and dumps the initial dates in there. If the file does exist, it reads the date out of it (only last line).
It then uses ~access/bin/calendar_more
to calculate the times for this run (initial date, end date, first date of next run, et cetera).
From this, it gets, amongst other things, the days_in_run
, which it then multiplies by 86,400 (secs per day) to get runtime
.
Next, the script creates it work and run directories, copying files there as needed. (Apparently, for historical runs, it also needs to change the dates in the UM files, changing the reference date from the 16th of the month to the first.) This is also where it uses the function it declared at the beginning.
Finally, it makes several (currently obscure) changes to a lot of namelists, and then creates the ACCESSRUNCMD
.
And it loads new modules, replacing others if they had been loaded before.
<RUNID>.fin
Check whether variable FCODE
is 0 (presumably the return code of the Model run?), only then will it do anything.
It calculates the name of the restart file restartfile="aiihca.da${umdate}"
where umdate
has been created by datetoum
or datetoum2
.
Next, it moves what I think are the coupling restart files, ${cplrundir}/?2?.nc
to the archive ${archivedir}/restart/cpl/$resfile-${enddate}
Then, it accesses $atmrundir
: Validates the date of the restart file, moves it to ${archivedir}/restart/atm/${restartarch
} ((restartarch
had been set to "${RUNID}.astart-${nextdate}"
).
Next, it moves all UM output files (aiihca.p*
) to the archive dir, renaming them in the process.
Modifications
Created a new branch: holger_testing
MOM5 Version Control
MOM5 is under version control: https://github.com/OceansAus/ACCESS-ESM1.5-MOM5.git
- this was added to the Makefile
.
ummodel_hg3
ummodel_hg3
was still copied from /short/p66/txz599/ACCESSHOME/submodels/UM/ummodel_hg3/
. That directory itself is under version control, but with some non-checked-in.
umbase_hg3
umbase_hg3
was still copied. But there weren't any interesting changes in the directory compared the the repository. (Only fcm_env.sh
and parsed_bld.cfg
) So I've changed it to point to the svn repo.
bld-hadgem3-mct.cfg
The file bld-hadgem3-mct.cfg
was missing from the ummodel_hg3
repo. For now, I've made it part of the access-esm repo, and added a line in the Makefile
to copy it over.