Difference between revisions of "Transition to Gadi"
m (Added MITgcm entry)
Line 1: Line 1:
 +
 
This page presents information specific to CLEx users and the status of the models supported by CMS.
 
This page presents information specific to CLEx users and the status of the models supported by CMS.
  
Line 4: Line 5:
  
 
== Progress ==
 
== Progress ==
−
This is the list of model versions that will be ported by us on Gadi.
+
 
−
Please contact [mailto:cws_help@nci.org.au cws_help@nci.org.au] if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision.
+
This is the list of model versions that will be ported by us on Gadi. Please contact [mailto:cws_help@nci.org.au cws_help@nci.org.au] if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision.
−
+
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! Model !! Status !! More Info & Performance Metrics
+
! Model
 +
! Status
 +
! More Info & Performance Metrics
 
|-
 
|-
−
| UM 7.3 (ACCESS AMIP 1.X) || WORKING ([[Transition_to_Gadi#ACCESS_1|Not Bit-Repro]]) || [[UM On Gadi#UM7.3]]
+
| UM 7.3 (ACCESS AMIP 1.X)
 +
| WORKING ([[Transition_to_Gadi#ACCESS_1|Not Bit-Repro]])
 +
| [[UM_On_Gadi#UM7.3|UM On Gadi#UM7.3]]
 
|-
 
|-
−
| UM 8.5 || IN PROGRESS ||
+
| UM 8.5
 +
| IN PROGRESS
 +
|  
 
|-
 
|-
−
| UM 10.4 (ACCESS AMIP 2) || TODO || [[UM On Gadi#UM10.4]]
+
| UM 10.4 (ACCESS AMIP 2)
 +
| TODO
 +
| [[UM_On_Gadi#UM10.4|UM On Gadi#UM10.4]]
 
|-
 
|-
−
| UM 11.4 (GA 7 / Nested) || WORKING || [[UM On Gadi#UM11.4]]
+
| UM 11.4 (GA 7 / Nested)
 +
| WORKING
 +
| [[UM_On_Gadi#UM11.4|UM On Gadi#UM11.4]]
 
|-
 
|-
−
| ACCESS-ESM 1.5 || IN PROGRESS  
+
| ACCESS-ESM 1.5
 +
| IN PROGRESS
 
|-
 
|-
−
| ACCESS-CM 2 || TODO
+
| ACCESS-CM 2
 +
| TODO
 
|-
 
|-
−
| ACCESS-CM 2 N48 || TODO
+
| ACCESS-CM 2 N48
 +
| TODO
 
|-
 
|-
−
| MOM || TODO
+
| MOM
 +
| TODO
 
|-
 
|-
−
| CABLE || TODO
+
| CABLE
 +
| TODO
 
|-
 
|-
−
| NU-WRF || TODO
+
| NU-WRF
 +
| TODO
 
|-
 
|-
−
| WRF 4.1.2 || TODO
+
| WRF 4.1.2
 +
| TODO
 
|-
 
|-
−
| WRF 4.1.1 || WORKING || [[WRF_v4.1.1_installation]]
+
| WRF 4.1.1
 +
| WORKING
 +
| [[WRF_v4.1.1_installation|WRF_v4.1.1_installation]]
 
|-
 
|-
−
| WRF 4.0.2 || TODO
+
| WRF 4.0.2
 +
| TODO
 
|-
 
|-
−
| WRF 3.9.1.1 Chem || TODO
+
| WRF 3.9.1.1 Chem
 +
| TODO
 
|-
 
|-
−
| WRF 3.9 || TODO
+
| WRF 3.9
 +
| TODO
 
|-
 
|-
−
| WRF 3.7.1 || TODO
+
| WRF 3.7.1
 +
| TODO
 
|-
 
|-
−
| WRF 3.6.1 || TODO
+
| WRF 3.6.1
 +
| TODO
 
|-
 
|-
−
| WRF 3.6 || TODO
+
| WRF 3.6
 +
| TODO
 
|-
 
|-
−
| WRF 3.5.1 || TODO
+
| WRF 3.5.1
 +
| TODO
 
|-
 
|-
−
| WPS data || Updated. See [[How_to_run_WRF]] for information.  
+
| WPS data
 +
| Updated. See [[How_to_run_WRF|How_to_run_WRF]] for information.  
 +
Also valid for runs on Raijin
  
−
Also valid for runs on Raijin
 
 
|-
 
|-
−
|}
+
|MITgcm
 +
| [[MITgcm on gadi|MITgcm on gadi]]
 +
}
  
 
=== Updates - Week of 25th Nov ===
 
=== Updates - Week of 25th Nov ===
Line 59: Line 89:
 
==== Messages from NCI ====
 
==== Messages from NCI ====
  
−
Nov 27 2019: Decomissioning of Portions of Normal/Express Queues
+
Nov 27 2019: Decomissioning of Portions of Normal/Express Queues Approximately half of the nodes servicing the normal/express queues have been decomissioned to allow for power reticulation works to comission Phase 2 of Gadi. Increased wait times for these queues are now likely.
−
Approximately half of the nodes servicing the normal/express queues have
 
−
been decomissioned to allow for power reticulation works to comission
 
−
Phase 2 of Gadi. Increased wait times for these queues are now likely.
 
  
 
We recommend that users migrate to Gadi as soon as possible.
 
We recommend that users migrate to Gadi as soon as possible.
  
 +
 
  
 
==== ACCESS 1 ====
 
==== ACCESS 1 ====
Line 71: Line 99:
 
ACCESS 1 AMIP jobs are working on Gadi, but they're not currently bit-reproducible under different processor decompositions. This may lead to crashes in some decompositions that go away if you change the processor layout. Performance is roughly twice as fast in walltime as Raijin's default nodes, however due to the accounting changes the SU cost of a run will be roughly the same as it was on Raijin. We recommend starting with a decomposition of 16 x 12 processors for ACCESS 1 runs at the default 'n96' resolution.
 
ACCESS 1 AMIP jobs are working on Gadi, but they're not currently bit-reproducible under different processor decompositions. This may lead to crashes in some decompositions that go away if you change the processor layout. Performance is roughly twice as fast in walltime as Raijin's default nodes, however due to the accounting changes the SU cost of a run will be roughly the same as it was on Raijin. We recommend starting with a decomposition of 16 x 12 processors for ACCESS 1 runs at the default 'n96' resolution.
  
−
Instructions for moving an existing ACCESS 1 AMIP job to Raijin are available at https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1 Please let us know if this doesn't work for your run - it may require some changes for specific configurations.
+
Instructions for moving an existing ACCESS 1 AMIP job to Raijin are available at [https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1 https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1] Please let us know if this doesn't work for your run - it may require some changes for specific configurations.
  
 
The ACCESS support team is cleaning up input files - '''please check you run has all the input files it needs before Jan 6th'''
 
The ACCESS support team is cleaning up input files - '''please check you run has all the input files it needs before Jan 6th'''
  
−
Working runs and performance information are available listed under http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3
+
Working runs and performance information are available listed under [http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3 http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3]
  
 
== Default project setting ==
 
== Default project setting ==
 +
 
When you first access Gadi, you should check your default project is set appropriately.
 
When you first access Gadi, you should check your default project is set appropriately.
  
Line 83: Line 112:
  
 
== Access to projects ==
 
== Access to projects ==
−
On Gadi, there will no public access to any project. Additionally, only members of a project can access this project's filesystem space in jobs submitted to the HPC nodes.  
+
 
 +
On Gadi, there will no public access to any project. Additionally, only members of a project can access this project's filesystem space in jobs submitted to the HPC nodes.
  
 
Some environments you may wish to join are (follow the link to join the project):
 
Some environments you may wish to join are (follow the link to join the project):
 +
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! Project Code !! Contact !! Description
+
! Project Code
 +
! Contact
 +
! Description
 
|-
 
|-
−
| [https://my.nci.org.au/mancini/project/hh5 hh5] || cws_help@nci.org.au || Conda environment
+
| [https://my.nci.org.au/mancini/project/hh5 hh5]
 +
| cws_help@nci.org.au
 +
| Conda environment
 
|-
 
|-
−
| [https://my.nci.org.au/mancini/project/sx70 sx70] || cws_help@nci.org.au || WRF Input Data
+
| [https://my.nci.org.au/mancini/project/sx70 sx70]
 +
| cws_help@nci.org.au
 +
| WRF Input Data
 
|-
 
|-
−
| [https://my.nci.org.au/mancini/project/access access] || cws_help@nci.org.au || ACCESS/UM Input Data
+
| [https://my.nci.org.au/mancini/project/access access]
 +
| cws_help@nci.org.au
 +
| ACCESS/UM Input Data
 
|-
 
|-
−
| [https://my.nci.org.au/mancini/project/xc57 xc57] || cws_help@nci.org.au || Permission to use VDI (only if you're not in a [[CLEX_projects_at_NCI|compute project]])
+
| [https://my.nci.org.au/mancini/project/xc57 xc57]
 +
| cws_help@nci.org.au
 +
| Permission to use VDI (only if you're not in a [[CLEX_projects_at_NCI|compute project]])
 
|}
 
|}
  
 
=== conda environments ===
 
=== conda environments ===
 +
 
If you are using the conda environments under /g/data/hh5/public/modules, you need to request membership of the hh5 project if not already done. [https://my.nci.org.au/mancini/project/hh5 Here].
 
If you are using the conda environments under /g/data/hh5/public/modules, you need to request membership of the hh5 project if not already done. [https://my.nci.org.au/mancini/project/hh5 Here].
  
 
== Models not supported by CMS ==
 
== Models not supported by CMS ==
−
You should try and update the library your model is using to the versions that will be installed on Gadi as much as possible.
 
−
Then test if your model is compiling and working with the updated libraries
 
  
 +
You should try and update the library your model is using to the versions that will be installed on Gadi as much as possible. Then test if your model is compiling and working with the updated libraries
 +
 +
 
 +
 +
== Climate Models supported by CMS ==
  
−
== Climate Models supported by CMS==
 
 
=== WRF ===
 
=== WRF ===
−
Some issues appeared when compiling WRF with OpenMPI v4.0.1. These are being worked out.
 
−
WRF is currently running on raijin with OpenMPI v2.1.3 so it is likely to run smoothly with other OpenMPI versions that will be installed on Gadi. We cannot test before access to Gadi.
 
  
−
/projects/WRF will disappear. It will be replaced by a new project: sx70. All users of WRF will need to be a member of sx70 to have access to the GEOG dataset. Although the project isn’t ready for use, you can start asking for membership here: https://my.nci.org.au/mancini/project/sx70
+
Some issues appeared when compiling WRF with OpenMPI v4.0.1. These are being worked out. WRF is currently running on raijin with OpenMPI v2.1.3 so it is likely to run smoothly with other OpenMPI versions that will be installed on Gadi. We cannot test before access to Gadi.
 +
 
 +
/projects/WRF will disappear. It will be replaced by a new project: sx70. All users of WRF will need to be a member of sx70 to have access to the GEOG dataset. Although the project isn’t ready for use, you can start asking for membership here: [https://my.nci.org.au/mancini/project/sx70 https://my.nci.org.au/mancini/project/sx70]
  
 
=== NUWRF ===
 
=== NUWRF ===
 +
 
NUWRF is being ported to Gadi with NCI's help. We expect a working solution will be ready shortly after Gadi is available.
 
NUWRF is being ported to Gadi with NCI's help. We expect a working solution will be ready shortly after Gadi is available.
  
Line 125: Line 170:
 
Please see the [https://accessdev.nci.org.au/trac/wiki/gadi#RoseCylcUMvn10ACCESS2orlater ACCESS wiki] for information
 
Please see the [https://accessdev.nci.org.au/trac/wiki/gadi#RoseCylcUMvn10ACCESS2orlater ACCESS wiki] for information
  
−
 
+
[[Category:Gadi]]
−
[[Category: Gadi]]
 

Revision as of 20:55, 3 December 2019

This page presents information specific to CLEx users and the status of the models supported by CMS.

The main source of information for Gadi is hosted on NCI help pages: Preparing for Gadi. This page will only list additional, specific information. Please continue to refer to NCI's pages for main information.

Progress

This is the list of model versions that will be ported by us on Gadi. Please contact cws_help@nci.org.au if you need us to port other versions. Note: we do not support CESM or any of its derivated models. It might not be possible to port all versions requested, any problematic request will be discussed with the Infrastructure Committee for a decision.

Model Status More Info & Performance Metrics
UM 7.3 (ACCESS AMIP 1.X) WORKING (Not Bit-Repro) UM On Gadi#UM7.3
UM 8.5 IN PROGRESS  
UM 10.4 (ACCESS AMIP 2) TODO UM On Gadi#UM10.4
UM 11.4 (GA 7 / Nested) WORKING UM On Gadi#UM11.4
ACCESS-ESM 1.5 IN PROGRESS
ACCESS-CM 2 TODO
ACCESS-CM 2 N48 TODO
MOM TODO
CABLE TODO
NU-WRF TODO
WRF 4.1.2 TODO
WRF 4.1.1 WORKING WRF_v4.1.1_installation
WRF 4.0.2 TODO
WRF 3.9.1.1 Chem TODO
WRF 3.9 TODO
WRF 3.7.1 TODO
WRF 3.6.1 TODO
WRF 3.6 TODO
WRF 3.5.1 TODO
WPS data Updated. See How_to_run_WRF for information.

Also valid for runs on Raijin

MITgcm MITgcm on gadi

}

Updates - Week of 25th Nov

Messages from NCI

Nov 27 2019: Decomissioning of Portions of Normal/Express Queues Approximately half of the nodes servicing the normal/express queues have been decomissioned to allow for power reticulation works to comission Phase 2 of Gadi. Increased wait times for these queues are now likely.

We recommend that users migrate to Gadi as soon as possible.

 

ACCESS 1

ACCESS 1 AMIP jobs are working on Gadi, but they're not currently bit-reproducible under different processor decompositions. This may lead to crashes in some decompositions that go away if you change the processor layout. Performance is roughly twice as fast in walltime as Raijin's default nodes, however due to the accounting changes the SU cost of a run will be roughly the same as it was on Raijin. We recommend starting with a decomposition of 16 x 12 processors for ACCESS 1 runs at the default 'n96' resolution.

Instructions for moving an existing ACCESS 1 AMIP job to Raijin are available at https://accessdev.nci.org.au/trac/wiki/gadi#UMUIUMvn7ACCESS1 Please let us know if this doesn't work for your run - it may require some changes for specific configurations.

The ACCESS support team is cleaning up input files - please check you run has all the input files it needs before Jan 6th

Working runs and performance information are available listed under http://climate-cms.wikis.unsw.edu.au/UM_On_Gadi#UM7.3

Default project setting

When you first access Gadi, you should check your default project is set appropriately.

Open the file: ~/.config/gadi-login.conf. Check the value of the PROJECT variable, set it to the project you want to use as default.

Access to projects

On Gadi, there will no public access to any project. Additionally, only members of a project can access this project's filesystem space in jobs submitted to the HPC nodes.

Some environments you may wish to join are (follow the link to join the project):

Project Code Contact Description
hh5 cws_help@nci.org.au Conda environment
sx70 cws_help@nci.org.au WRF Input Data
access cws_help@nci.org.au ACCESS/UM Input Data
xc57 cws_help@nci.org.au Permission to use VDI (only if you're not in a compute project)

conda environments

If you are using the conda environments under /g/data/hh5/public/modules, you need to request membership of the hh5 project if not already done. Here.

Models not supported by CMS

You should try and update the library your model is using to the versions that will be installed on Gadi as much as possible. Then test if your model is compiling and working with the updated libraries

 

Climate Models supported by CMS

WRF

Some issues appeared when compiling WRF with OpenMPI v4.0.1. These are being worked out. WRF is currently running on raijin with OpenMPI v2.1.3 so it is likely to run smoothly with other OpenMPI versions that will be installed on Gadi. We cannot test before access to Gadi.

/projects/WRF will disappear. It will be replaced by a new project: sx70. All users of WRF will need to be a member of sx70 to have access to the GEOG dataset. Although the project isn’t ready for use, you can start asking for membership here: https://my.nci.org.au/mancini/project/sx70

NUWRF

NUWRF is being ported to Gadi with NCI's help. We expect a working solution will be ready shortly after Gadi is available.

UM with UMUI (e.g. ACCESS 1.X AMIP )

Please see the ACCESS wiki for information

UM with Rose/Cylc (e.g. ACCESS 2, GA)

Please see the ACCESS wiki for information

Retrieved from "http://climate-cms.wikis.unsw.edu.au/index.php?title=Transition_to_Gadi&oldid=1617"