High Res MIP
Contents
Global Atmosphere 7.1 N216
To compare the performance on our systems with the MetOffice results, we have created a local version of the MetOffice's GA7.1 N216 model, specifically of their High Resolution Model Intercomparison Project. The ID of this rose is u-an493}}, which is a copy of the MetOffice's {{u-ai718, with only very few changes to make it compatible with NCI raijin:
- The paths to the ancillary files were adjusted
- The number of CPUs was changed to 26x32 (with 2OMP threads)
- The submission scripts were adapted for NCI (PBS, modules, etc)
Performance
- There are two recommended decompositions:
- By default, the job runs on 24 nodes (16 cores each), specifically 24x16, with no OpenMP. This is a reasonable compromise between speed (~28 hr/yr) and cost (~11kSU/yr)
- The fastest decomposition is 26x32, with 2 OMP threads, or 104 nodes. Adding more CPUs only increases overhead. It can model a year in about 11 hours, but at a cost of 20kSU per model year.
- The resubmission frequency is set to 1 month, and should not be changed to less than 10 days to not impact performance.
Obtaining the Job
As the rose is located on the MetOffice Collaboration Repository, you need an account there [link missing].
$ mosrs-auth
$ access-auth
$ rosie copy u-an493
Notes
- The metadata for the UM is located in /g/data/access/AMEL/GA7.1-HRMIP/rose-meta/um-atmos/HEAD}}, which is under version control at {{https://metoffice-mirror/svn/um/main/branches/dev/scottwales/r31906_easyaerosol_raijin/rose-meta/um-atmos/HEAD
- In the Machine Options section there is a definition of a variable HRMIP_DIR}} which is a path prefix to keep the actual filenames for the MetOffice as little changed as possible. Currently it points to a location under {{/g/data1/w35/saw562/, but this might have to change in the future.
External Links
http://collab.metoffice.gov.uk/twiki/bin/view/Project/HiResCL/HiResCLPRIMSimulations https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIP https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIP/highresSST-present/HadGEM3-GC31-N216-ORCA025 https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIPForcings https://code.metoffice.gov.uk/trac/roses-u/log/a/n/4/9/3/trunk
Global Atmosphere 7.1 N96
A version of HighResMIP is also available at N96 resolution, using the job u-as709.
As in the n216 version, only minor changes were made to the model's data paths (Local copies of the files used at the Met Office can be found under /g/data1/w35/saw562/HighResMip/data)
On Raijin's 'normal' queue with 16x16 processors this costs ~ 600 SU per model year, and takes ~ 6 hours per model year to run.
Transpose AMIP style N216 runs
Transpose AMIP style hindcast runs can be run using the job u-ap508. This allows researchers to set the model start date to an arbitrary time and focus on specific weather and climate events.
This configuration is based on the HighResMIP configuration, but uses a Gregorian calendar rather than the original 360 day calendar.
Initialised fields
The following fields are sourced from the ERA Interim GRIB datasets under /g/data1/ub4. Details of how this works can be found in | UMDP 303
ECMWF short name | Description |
z | Geopotential height |
lsm | Land sea mask |
sp | Surface pressure |
skt | Skin temperature |
sd | Snow depth |
stl1 | Soil temperature by layer |
stl2 | |
stl3 | |
stl4 | |
u | U wind |
v | V wind |
t | Temperature |
q | Specific humidity |
Additionally a number of model fields required by the model are initialised to zero, and require some spin-up time
STASH code | Description |
0155 | Accumulated surface runoff |
0156 | Accumulated sub-surface runoff |
0157 | Gridbox area |
0290 | Daily accumulated lake flux |
0511 | Inland basin flow |
Updated Fields
Sea surface temperature and sea ice are updated from ERA-Interim at 6 hourly intervals. The ancillary files use for this are generated by the `install_ancil}}` task, they are regridded to the model resolution using NCO then converted to UM format with {{mkancil.
The `install_ancil` task also sets up the calendar for EasyAerosol input files. As these files are large (the total size is ~1TB) the script first extracts just the dates important to the run before changing the calendar.
External Links
| UMDP 303: Reading GRIB data using the UM | Roff et al. 2012 - Initializing ACCESS from Era-interim data
include component="page" wikiName="climate-cms" page="Index UM" editable="1"