climate-cms wikis.unsw.edu.au - User contributions [en]

AusCOM

2013-03-05T02:19:10Z

Marshallward: Imported from Wikispaces

ACCESS-O (or AusCOM) is the ocean-ice implementation of the ACCESS climate model. It consists of the ocean (MOM) and ice (CICE) components coupled through OASIS and a data-driven atmospheric model (MATM), primarily driven by CORE atmospheric forcing.

This document provides basic instructions for building and running standard CORE-forced experiments.

''Note: These instructions assume that the user has access to the v45 project. Contact CMS if you do not belong to this project.''

= Installing AusCOM =

First locate an appropriate directory from where you can run AusCOM (usually /short) and create a directory to hold your source code, e.g.:
<syntaxhighlight lang=bash>
mkdir -p /short/v45/${USER}/projects # Or any directory
</syntaxhighlight>
where ${USER} is your username.

''Note: Do not use <nowiki>/short/v45/${USER}/auscom</nowiki>, since model output is stored in this directory.''

Next, clone a copy of the repository:
<syntaxhighlight lang=bash>
cd /short/v45/${USER}/projects
git clone http://github.com/coecms/auscom.git
</syntaxhighlight>
Once this complete, run the build script:
<syntaxhighlight lang=bash>
cd auscom
./build_auscom.sh
</syntaxhighlight>
A full build of all AusCOM components will usually take about an hour.

Once installation is complete, run the link_input.sh script to create symbolic links to the input forcing fields:
<syntaxhighlight lang=bash>
./link_input.sh
</syntaxhighlight>
The original copies of these files are stored in /g/data/v45/auscom. If you want to modify the forcing fields and configuration files, then you will need to create new symbolic links to your own directories. ''Contact CMS if you require any assistance.''

You should now be ready to submit an AusCOM experiment.

However, if you ever need to delete the executables and rebuild AusCOM, there is a script named clean_auscom.sh script which will delete any compiled binaries:
<syntaxhighlight lang=bash>
./clean_auscom.sh
</syntaxhighlight>

= Running AusCOM =

The default experiment cnyf2-sw1 is located with the repository. To run the experiment, move to its directory and run the provided script:
<syntaxhighlight lang=bash>
cd exp/cnyf2-sw1
qsub run_auscom.sh
</syntaxhighlight>
This will run for one model year, and will take approximately 3 hours.

Model output can be accessed in the output/cnyf2-sw1/history directory.

''This is a symbolic link to the actual data, located at /short/v45/${USER}/auscom/OUTPUT''

= Configuring AusCOM =

AusCOM is configured to run a series of self-resubmitting one-year simulations.

The total duration is set within the run_auscom.sh script. In this part of this script (line 135):
<syntaxhighlight lang=bash>
# Initial and final date of the experiment
if [[ $DEBUG = "yes" ]]; then
iniyear=1; finalyear=1; typeset -Z4 iniyear finalyear
inimonth=1; finalmonth=1; typeset -Z2 inimonth finalmonth
iniday=1; finalday=4; typeset -Z2 iniday finalday
else
iniyear=1; finalyear=5; typeset -Z4 iniyear finalyear
inimonth=1; finalmonth=12; typeset -Z2 inimonth finalmonth
iniday=1; finalday=31; typeset -Z2 iniday finalday
fi
</syntaxhighlight>
Under the else}} block (non-debug mode), replace {{finalyear with the numbers of years that you want to run. (Set as five years in this example).

To change the duration of a single run, locate this code block (line 146):
<syntaxhighlight lang=bash>
# Duration of this run (maybe the most often visited place for test/short runs):
if [[ $DEBUG = "yes" ]]; then
nyear=0 # number of years (ALWAYS 0 ! change nmonth etc...)
nmonth=0 # number of months
nday=1 # number of days
else
nyear=0 # number of years (ALWAYS 0 ! change nmonth etc...)
nmonth=12 # number of months
nday=0 # number of days
fi
</syntaxhighlight>
and replace nmonth}} and {{ndays}} of the {{else block with the desired duration.

Standard PBS configuration also applies to the script header (beginning on line 35 of run_auscom.sh). Most have been pre-configured for the standard one-year experiment, but users may need to change the following:

* #PBS -P v45}}: Deduct CPU hours from project {{v45
* #PBS -W group_list=v45}}: New files belong to the {{v45 group
* #PBS -l walltime=4:00:00: Submit a four-hour job

If you need further information about model configuration, contact CMS.

MOM output diagnostics

2012-07-19T07:17:39Z

Marshallward: Imported from Wikispaces

MOM output is managed through the FMS diagnostics manager, and configured using the diag_table}} file. This page describes the format of {{diag_table records and how to manage output diagnostic in MOM.

= diag_table =

diag_table configuration files contain two types of records:

* Files: NetCDF output files containing MOM4 output
* Fields: Individual data fields (such as velocity or temperature) to be saved.

== Output file specification ==

diag_table files begin with output file records. An example output file is shown below:
<syntaxhighlight lang=fortran>
"ocean_snap", 5, "days", 1, "days", "Time"
</syntaxhighlight>
Each field is discussed below (in order of appearance)

* "ocean_snap"}}: The filename of a target netCDF output file. A {{.nc extension will be appended to the filename.
* 5, "days": The output frequency for variables in this file, followed by units. Standard calendar units can be used (days, hours, etc.).
* 1, "days": Output time axis unit of measurement (with units).
* "Time": Time axis variable name in the netCDF file.

Output frequency and other time-related information is associated with individual files. For this reason, variables with similar output rates should generally be grouped together in the same file.

== Field specification ==

Example field record:
<syntaxhighlight lang=fortran>
"ocean_model", "u", "u", "ocean_snap", "all", .false., "none", 2
</syntaxhighlight>
* "ocean_model"}}: MOM4 module containing the diagnostic variable. This is typically attributed to a particular submodel, such as {{"ocean_model"}} or {{"ice_model".
* "u", "u": The MOM4 registered variable name, and the output netCDF variable name, respectively. The first must match the variable name in the MOM4 source code, but the second can be renamed by the user.
* "ocean_snap"}}: The output file (defined in the {{diag_table file record) which will store this variable.
* "all"}}: The sampling rate of the variable. {{"all" means that the variable is sampled for every value on the time axis (as specified in the file record).
* .false.}}: Time averaging between output times. {{.true.}} saves the time-averaged values between time sampling rates, while {{.false. returns snapshots of the fields at each point on the time axis.
* "none"}}: Optional arguments, usually defined in MOM4. {{"none" denotes no optional arguments.
* 2: Output data type. Four formats are supported:
** 1: Double-precision (64-bit) floating point
** 2: Single-precision (32-bit) floating point
** 4: 16-bit integer packed variables (typically used for long-term archival)
** 8: 8-bit integer-packed variables (untested)

= Creating new variables =

To create new variables for output not included in the default version requires a modification of the MOM4 source code.

Python for Scientists

2012-01-12T22:23:49Z

Marshallward: Imported from Wikispaces

=Python Course Outline=
''Note: This is an outline for a future course currently in development.''

==Introduction to Python==
===Installation===
Not intended as a CMS support topic, only how to quickly get started
* Enthought (self-contained python/numpy/scipy/matplotlib installation)
* Distribution packages (apt-get, yum, MacPorts)

===Starting Python===
* Python's interactive shell
* Python scripting (#!/usr/bin/env python stuff)
* iPython, PyLab: Interactive Matlab-like environments
* Spyder: A python-based GUI to emulate Matlab (not recommended?)

===Basics===
Focus on major differences from Matlab, programming languages:

* Forced indentation
* Built in data types: bool, int, float, complex, strings, lists, tuples, dicts, files, ...
* Indexing and Slicing (0..N-1, -1, -2, etc.)
* Aliasing: variables as labels
* Conditional syntax (if-then-else, while, etc.)
* Iterators: Looping without indexing ('for x in list:')
** Possibly some itertools examples?
* Functions
* Simple string manipulation
* Objects: Not OO design, just the persistence of objects
** e.g. anything (variables, functions, etc.) can be function arguments

===Modules===
* Using modules (import xyz, import xyz as x, from xyz import y)
* There are lots of modules for any given task, we should settle on a few recommendations (but also not be afraid to change our minds in the future)

==Numpy==
===Numpy vs Matlab===
* numpy arrays vs. Matlab matrices
* Array generation and manipulation (arange, linspace, zeros, meshgrid)
Review basic operations:
* Arithmetic on arrays
* built-in functions (sqrt, sin, etc.) and constants (pi)
* Numpy array indexing (syntax, slicing, etc.)
* Basic manipulations (concatenation, reshaping, tiling, vstack/hstack, etc.)

===Performance===
* Introduce numpy as a series of job submissions to the C libraries
* Vectorisation arithmetic (avoiding index loops)
* An Intel MKL numpy is often several times faster than a gcc BLAS numpy (though never actually confirmed this)
* numexpr: An easy-to-use, high-performance alternative to numpy for certain tasks, includes limited parallelisation options.

===Masked arrays (numpy.ma)===
(I have only some limited experience with ma, but it is very useful for land masks in CMIP ocean output)
* Mask creation (inc. logical numpy operators)
* Masked array manipulation

==SciPy==
I use scipy in a very ad-hoc manner, I don't have a comprehensive grasp of its features. But the following come to mind (with examples):

* Interpolation (scipy.interpolate)
** Earth grid generation
* Statistics (scipy.stats, scipy.random)
* Signal processing (scipy.signal)
** Time series analysis, filtering
* Linear Algebra
** SVD, EOF/PC analysis

==Image Analysis==
''Note: ANU GFD has a strong fluid dynamics laboratory, and CoE researcher Andy Hogg often supervises students in lab experiments.''

===Python Image Library (PIL)===

* I have no experience with PIL, but it has been used by at least one student here. It is a natural alternative to Matlab's builtin image analysis tools.
* I don't regard image analysis as CMS work, but including it in a course would help promote collaboration here at ANU.

==I/O==
===NetCDF===
* scipy.io.netcdf:
** Good performance
** Included with scipy
** imperfect NetCDF implementation (occasional garbage 'inf' data)
* netcdf4-python:
** Complete(?) NetCDF3/4 support
** Reduced performance
** Nontrivial installation

===ASCII/raw text input===
* I haven't done this in python, but there is still an occasional need for this (esp. for old data sets)

===pydap===
* Simplified interface to access netcdf files via OPeNDAP

===PyTables===
* HDF5 support, good performance
* (I haven't used this much)

==Plotting==
===Matplotlib (2D plotting)===

* Line/curve plots
* Scatter plots
* Field plots: Contours, image maps, etc.
* Subplots
* Frills (labels, legends, arrows, etc.)

===Mayavi (3D plotting)===
* I have never used this, but it's the only option that I know of

==Shell Interface (OS)==
The ANU group uses python for job submissions of numerical models on vayu, so there may be some interest in how to run subprocesses and manipulate files through python scripts, as if it were a traditional shell script.

* os, sys, shutils
* subprocess

==Earth Science Tools and Modules==
===basemap (matplotlib)===
* geographic plotting
* similar to mmap in Matlab

===datetime, calendar===
* Useful for calendar tracking

===gridding?===
* Are there generic gridding modules in python? e.g. mercator, cubic sphere, tripolar, etc?