Difference between revisions of "Transition to Gadi"
A.heerdegen (talk | contribs) m (Added MITgcm entry) |
A.heerdegen (talk | contribs) m (Fixed typo) |
||
Line 3: | Line 3: | ||
The main source of information for Gadi is hosted on NCI help pages: [https://opus.nci.org.au/display/Help/Preparing+for+Gadi Preparing for Gadi]. This page will only list additional, specific information. Please continue to refer to NCI's pages for main information. | The main source of information for Gadi is hosted on NCI help pages: [https://opus.nci.org.au/display/Help/Preparing+for+Gadi Preparing for Gadi]. This page will only list additional, specific information. Please continue to refer to NCI's pages for main information. | ||
+ | |||
== Progress == | == Progress == | ||
This is the list of model versions that will be ported by us on Gadi. Please contact [mailto:cws_help@nci.org.au cws_help@nci.org.au] if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision. | This is the list of model versions that will be ported by us on Gadi. Please contact [mailto:cws_help@nci.org.au cws_help@nci.org.au] if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision. | ||
+ | } | ||
+ | === Updates - Week of 25th Nov === | ||
+ | |||
+ | ==== Messages from NCI ==== | ||
+ | |||
+ | Nov 27 2019: Decomissioning of Portions of Normal/Express Queues Approximately half of the nodes servicing the normal/express queues have been decomissioned to allow for power reticulation works to comission Phase 2 of Gadi. Increased wait times for these queues are now likely. | ||
+ | |||
+ | We recommend that users migrate to Gadi as soon as possible. | ||
+ | |||
+ | | ||
+ | |||
+ | ==== ACCESS 1 ==== | ||
+ | |||
+ | ACCESS 1 AMIP jobs are working on Gadi, but they're not currently bit-reproducible under different processor decompositions. This may lead to crashes in some decompositions that go away if you change the processor layout. Performance is roughly twice as fast in walltime as Raijin's default nodes, however due to the accounting changes the SU cost of a run will be roughly the same as it was on Raijin. We recommend starting with a decomposition of 16 x 12 processors for ACCESS 1 runs at the default 'n96' resolution. | ||
+ | |||
+ | Instructions for moving an existing ACCESS 1 AMIP job to Raijin are available at [https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1 https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1] Please let us know if this doesn't work for your run - it may require some changes for specific configurations. | ||
+ | |||
+ | The ACCESS support team is cleaning up input files - '''please check you run has all the input files it needs before Jan 6th''' | ||
+ | |||
+ | Working runs and performance information are available listed under [http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3 http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3] | ||
{| class="wikitable" | {| class="wikitable" | ||
Line 81: | Line 102: | ||
|- | |- | ||
− | |MITgcm | + | | MITgcm |
− | | [[ | + | | [[MITgcm_on_gadi|MITgcm on gadi]] |
− | } | + | |} |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== Default project setting == | == Default project setting == |
Revision as of 20:56, 3 December 2019
This page presents information specific to CLEx users and the status of the models supported by CMS.
The main source of information for Gadi is hosted on NCI help pages: Preparing for Gadi. This page will only list additional, specific information. Please continue to refer to NCI's pages for main information.
Contents
Progress
This is the list of model versions that will be ported by us on Gadi. Please contact cws_help@nci.org.au if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision. }
Updates - Week of 25th Nov
Messages from NCI
Nov 27 2019: Decomissioning of Portions of Normal/Express Queues Approximately half of the nodes servicing the normal/express queues have been decomissioned to allow for power reticulation works to comission Phase 2 of Gadi. Increased wait times for these queues are now likely.
We recommend that users migrate to Gadi as soon as possible.
ACCESS 1
ACCESS 1 AMIP jobs are working on Gadi, but they're not currently bit-reproducible under different processor decompositions. This may lead to crashes in some decompositions that go away if you change the processor layout. Performance is roughly twice as fast in walltime as Raijin's default nodes, however due to the accounting changes the SU cost of a run will be roughly the same as it was on Raijin. We recommend starting with a decomposition of 16 x 12 processors for ACCESS 1 runs at the default 'n96' resolution.
Instructions for moving an existing ACCESS 1 AMIP job to Raijin are available at https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1 Please let us know if this doesn't work for your run - it may require some changes for specific configurations.
The ACCESS support team is cleaning up input files - please check you run has all the input files it needs before Jan 6th
Working runs and performance information are available listed under http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3
Model | Status | More Info & Performance Metrics |
---|---|---|
UM 7.3 (ACCESS AMIP 1.X) | WORKING (Not Bit-Repro) | UM On Gadi#UM7.3 |
UM 8.5 | IN PROGRESS | |
UM 10.4 (ACCESS AMIP 2) | TODO | UM On Gadi#UM10.4 |
UM 11.4 (GA 7 / Nested) | WORKING | UM On Gadi#UM11.4 |
ACCESS-ESM 1.5 | IN PROGRESS | |
ACCESS-CM 2 | TODO | |
ACCESS-CM 2 N48 | TODO | |
MOM | TODO | |
CABLE | TODO | |
NU-WRF | TODO | |
WRF 4.1.2 | TODO | |
WRF 4.1.1 | WORKING | WRF_v4.1.1_installation |
WRF 4.0.2 | TODO | |
WRF 3.9.1.1 Chem | TODO | |
WRF 3.9 | TODO | |
WRF 3.7.1 | TODO | |
WRF 3.6.1 | TODO | |
WRF 3.6 | TODO | |
WRF 3.5.1 | TODO | |
WPS data | Updated. See How_to_run_WRF for information.
Also valid for runs on Raijin | |
MITgcm | MITgcm on gadi |
Default project setting
When you first access Gadi, you should check your default project is set appropriately.
Open the file: ~/.config/gadi-login.conf. Check the value of the PROJECT variable, set it to the project you want to use as default.
Access to projects
On Gadi, there will no public access to any project. Additionally, only members of a project can access this project's filesystem space in jobs submitted to the HPC nodes.
Some environments you may wish to join are (follow the link to join the project):
Project Code | Contact | Description |
---|---|---|
hh5 | cws_help@nci.org.au | Conda environment |
sx70 | cws_help@nci.org.au | WRF Input Data |
access | cws_help@nci.org.au | ACCESS/UM Input Data |
xc57 | cws_help@nci.org.au | Permission to use VDI (only if you're not in a compute project) |
conda environments
If you are using the conda environments under /g/data/hh5/public/modules, you need to request membership of the hh5 project if not already done. Here.
Models not supported by CMS
You should try and update the library your model is using to the versions that will be installed on Gadi as much as possible. Then test if your model is compiling and working with the updated libraries
Climate Models supported by CMS
WRF
Some issues appeared when compiling WRF with OpenMPI v4.0.1. These are being worked out. WRF is currently running on raijin with OpenMPI v2.1.3 so it is likely to run smoothly with other OpenMPI versions that will be installed on Gadi. We cannot test before access to Gadi.
/projects/WRF will disappear. It will be replaced by a new project: sx70. All users of WRF will need to be a member of sx70 to have access to the GEOG dataset. Although the project isn’t ready for use, you can start asking for membership here: https://my.nci.org.au/mancini/project/sx70
NUWRF
NUWRF is being ported to Gadi with NCI's help. We expect a working solution will be ready shortly after Gadi is available.
UM with UMUI (e.g. ACCESS 1.X AMIP )
Please see the ACCESS wiki for information
UM with Rose/Cylc (e.g. ACCESS 2, GA)
Please see the ACCESS wiki for information