High Res MIP

Global Atmosphere 7.1 N216

To compare the performance on our systems with the MetOffice results, we have created a local version of the MetOffice's GA7.1 N216 model, specifically of their High Resolution Model Intercomparison Project. The ID of this rose is u-an493}}, which is a copy of the MetOffice's {{u-ai718, with only very few changes to make it compatible with NCI raijin:

  • The paths to the ancillary files were adjusted
  • The number of CPUs was changed to 26x32 (with 2OMP threads)
  • The submission scripts were adapted for NCI (PBS, modules, etc)

Performance

  • There are two recommended decompositions:
    • By default, the job runs on 24 nodes (16 cores each), specifically 24x16, with no OpenMP. This is a reasonable compromise between speed (~28 hr/yr) and cost (~11kSU/yr)
    • The fastest decomposition is 26x32, with 2 OMP threads, or 104 nodes. Adding more CPUs only increases overhead. It can model a year in about 11 hours, but at a cost of 20kSU per model year.
  • The resubmission frequency is set to 1 month, and should not be changed to less than 10 days to not impact performance.

Obtaining the Job

As the rose is located on the MetOffice Collaboration Repository, you need an account there [link missing].

$ mosrs-auth
$ access-auth
$ rosie copy u-an493

Notes

External Links

http://collab.metoffice.gov.uk/twiki/bin/view/Project/HiResCL/HiResCLPRIMSimulations https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIP https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIP/highresSST-present/HadGEM3-GC31-N216-ORCA025 https://code.metoffice.gov.uk/trac/ukcmip6/wiki/CMIP6HighResMIPForcings https://code.metoffice.gov.uk/trac/roses-u/log/a/n/4/9/3/trunk

Global Atmosphere 7.1 N96

A version of HighResMIP is also available at N96 resolution, using the job u-as709.

As in the n216 version, only minor changes were made to the model's data paths (Local copies of the files used at the Met Office can be found under /g/data1/w35/saw562/HighResMip/data)

On Raijin's 'normal' queue with 16x16 processors this costs ~ 600 SU per model year, and takes ~ 6 hours per model year to run.

Transpose AMIP style N216 runs

Transpose AMIP style hindcast runs can be run using the job u-ap508. This allows researchers to set the model start date to an arbitrary time and focus on specific weather and climate events.

This configuration is based on the HighResMIP configuration, but uses a Gregorian calendar rather than the original 360 day calendar.

Initialised fields

The following fields are sourced from the ERA Interim GRIB datasets under /g/data1/ub4. Details of how this works can be found in | UMDP 303

ECMWF short name Description
z Geopotential height
lsm Land sea mask
sp Surface pressure
skt Skin temperature
sd Snow depth
stl1 Soil temperature by layer
stl2
stl3
stl4
u U wind
v V wind
t Temperature
q Specific humidity

Additionally a number of model fields required by the model are initialised to zero, and require some spin-up time

STASH code Description
0155 Accumulated surface runoff
0156 Accumulated sub-surface runoff
0157 Gridbox area
0290 Daily accumulated lake flux
0511 Inland basin flow

Updated Fields

Sea surface temperature and sea ice are updated from ERA-Interim at 6 hourly intervals. The ancillary files use for this are generated by the `install_ancil}}` task, they are regridded to the model resolution using NCO then converted to UM format with {{mkancil.

The `install_ancil` task also sets up the calendar for EasyAerosol input files. As these files are large (the total size is ~1TB) the script first extracts just the dates important to the run before changing the calendar.

External Links

| UMDP 303: Reading GRIB data using the UM | Roff et al. 2012 - Initializing ACCESS from Era-interim data

include component="page" wikiName="climate-cms" page="Index UM" editable="1"