S.wales at 03:34, 12 December 2019

2019-12-12T03:34:08Z

S.wales at 04:33, 5 December 2019

2019-12-05T04:33:40Z

S.wales at 05:22, 12 March 2019

2019-03-12T05:22:58Z

ScottWales: Imported from Wikispaces

2013-11-01T06:35:25Z

Imported from Wikispaces

New page

=Installation notes for UM 8.5=

This roughly follows the order of the external release documentation, and notes any differences on Raijin, as well as stylistic differences between the Met Office and the Australian community.

On the NCI system we have a central UMUI server called '''accesscollab''', researchers connect to this via ssh to run a UMUI variant developed at CSIRO called UMUIX. Running the UMUI directly on users desktops is not supported, both because our licence disallows it and because of the difficulty of supporting it in a non-centralised environment.

The installation at NCI is used by researchers from the CoE, CSIRO and BoM. CSIRO and the Bureau also have their own separate UMUI servers that can submit jobs to Raijin.

At NCI the UM is supported by a joint team of CoE, CSIRO and BoM staff under the umbrella of 'ACCESS'.

The installation of UM8.5 on Raijin was done by Scott Wales at the CoE.

==1.1 Rose Migration==

Preparations are being made at NCI to support the new Rose UI on a new 'cloud' server, this will be similar to accesscollab in that it's a central UI server that all users log in to.

==2 Prerequisites==

At NCI we make extensive use of the software module system to provide pre-requisite software libraries as well as tools for data analysis. Pre-requisites are automatically loaded by the UM run scripts, to see what is available run
<syntaxhighlight>
module use ~access/modules
module list
</syntaxhighlight>

===2.4 Shells===
The UMUI scripts are written to use the Korn shell (ksh). This is fairly similar to Bash syntax, it's fairly rare to find in Linux environments since historically it wasn't open source. Note that the CMS team does not provide support for using this as a login shell; supported shells at NCI are bash and tcsh.

===2.5 Remote access===
Remote access in the NCI environment is done exclusively through ssh. We recommend you set up a ssh key & passphrase on your local desktop and copy the public key to Raijin and accesscollab, this way your desktop can save the key and you don't have to enter a passphrase each time you log in.

==3 Resources==

The external release test job is available in the UMUI as job '''[todo]'''

==4 Preparation==

Source code for the UM systems is available at NCI on the access-svn server. The following repositories contain UM related code:

* https://access-svn.nci.org.au/svn/um
* https://access-svn.nci.org.au/svn/umui
* https://access-svn.nci.org.au/svn/UM_Admin
* https://access-svn.nci.org.au/svn/GCOM
* https://access-svn.nci.org.au/svn/ghui
* https://access-svn.nci.org.au/svn/jules

In each of these repositories 'trunk' is kept as a mirror of the Met Office trunk, local changes are kept in branches.

In particular the main UM branch is

* https://access-svn.nci.org.au/svn/um/branches/dev/vn8.5/local_changes

The local changes primarily relate to differences in build settings & NCI's job submission system, any patches released by the Met Office between releases also get merged here (as well as the branch 'metoffice_patches').

==6 FCM==

FCM is a wrapper around subversion for providing shortcuts to repository paths (e.g. fcm:um_tr instead of https://access-svn.nci.org.au/svn/um/trunk) as well as replacements for some subversion functions (e.g. branch, merge) to suit Met Office workflows. It is also a build system (ala 'make') because why not. It has advantages and disadvantages, if you want to use normal subversion commands to work with repositories feel free. The build system is likewise rather opaque in what it does behind the scenes.

Following repos updated:

* UM_Admin
* UMUI
* UM
* JULES

Patches & local modifications for UM at

https://access-svn.nci.org.au/trac/um/browser/branches/dev/vn8.5

Post-release patches from the Met Office are in the metoffice_patches branch; local changes are in local_changes (mostly compiler settings)

The UMUI changes to support PBS Pro on Raijin done for 8.4 were merged into the 8.5 UMUI folder

The JULES update was missing a patch, Met Office released update

The UmInstall script does the following in a somewhat convoluted & obscured fashion:

<syntaxhighlight>
AUXREPO= # SVN repository for AUX
UMUIREPO= # SVN repository for UMUI

VN=8.5
PLATNAME=linux

CTLDATA=$UMDIR/vn$VN/ctldata
PLATFORM=$UMDIR/vn$VN/$PLATNAME

svn export $ADMINREPO/Install $UMDIR/vn$VN
mv $UMDIR/vn$VN/PLATFORM $PLATFORM

svn export $AUXREPO $CTLDATA

mkdir $CTLDATA/ANCILmaster
mkdir $CTLDATA/STASHmaster
svn export $UMUIREPO/vn$VN/variables/AncilFields_A $CTLDATA/ANCILmaster/AncilFields_A
svn export $UMUIREPO/vn$VN/variables/AncilFiles_A $CTLDATA/ANCILmaster/AncilFiles_A
svn export $UMUIREPO/vn$VN/variables/STASHmaster/STASHmaster_A $CTLDATA/ANCILmaster/AncilFiles_A

svn export $UMREPO/src/script/functions $PLATFORM/functions

for file in $PLATFORM/scripts/*; do
sed -i $file -e 's/\@UMMACHINE\@/'$HOSTNAME'/'
sed -i $file -e 's/\@UM_AWK\@/awk/'
sed -i $file -e 's/\@UM_SED\@/sed/'
sed -i $file -e 's/\@UM_GREP\@/grep/'
sed -i $file -e 's/\@UMDIR\@/'$UMDIR'/'
sed -i $file -e 's/\@VN_PATH\@/vn'$VN'/'
sed -i $file -e 's/\@NCOMP_PROCS\@/1/'
done

mv $PLATFORM/scripts/.umsetvars $PLATFORM/scripts/.umsetvars_vn$VN

chmod +x $PLATFORM/scripts

</syntaxhighlight>

This was done inside the UM_Admin repository as a 8.5 local changes branch.

Prebuilds were done by processing the sample job in the UMUI then manually changing the extract and build target directories in the UMUI scripts to ~access/prebuilds/vn8.5_safe on collab and $UMDIR/prebuilds/vn8.5_safe on raijin

Once prebuilds are built need to delete 'bin' folder from each directory so scripts & recon get built properly

← Older revision		Revision as of 04:33, 5 December 2019
Line 121:		Line 121:
	Once prebuilds are built need to delete 'bin' folder from each directory so scripts & recon get built properly		Once prebuilds are built need to delete 'bin' folder from each directory so scripts & recon get built properly

−	[[Category:~~Unified Model~~]]	+	[[Category:Admin]]

← Older revision		Revision as of 05:22, 12 March 2019
Line 120:		Line 120:

	Once prebuilds are built need to delete 'bin' folder from each directory so scripts & recon get built properly		Once prebuilds are built need to delete 'bin' folder from each directory so scripts & recon get built properly
		+
		+	[[Category:Unified Model]]

@@ Line 18: / Line 18: @@
 At NCI we make extensive use of the software module system to provide pre-requisite software libraries as well as tools for data analysis. Pre-requisites are automatically loaded by the UM run scripts, to see what is available run
-<syntaxhighlight>
+<syntaxhighlight lang=bash>
 module use ~access/modules
 module list
@@ Line 76: / Line 76: @@
 The UmInstall script does the following in a somewhat convoluted & obscured fashion:
-<syntaxhighlight>
+<syntaxhighlight lang=bash>
 AUXREPO= # SVN repository for AUX
 UMUIREPO= # SVN repository for UMUI

Installing um8.5 - Revision history

S.wales at 03:34, 12 December 2019

S.wales at 04:33, 5 December 2019

S.wales at 05:22, 12 March 2019

ScottWales: Imported from Wikispaces