Difference between revisions of "Override Files"

 
(One intermediate revision by the same user not shown)
Line 3: Line 3:
  
 
The FCM build script is created using a hierarchy of config scripts, some of which are from SVN, some from the UMUI output and some from override files. The basic structure of the output is given by container.cfg, which starts the include process. Variables are declared using a percent sign,
 
The FCM build script is created using a hierarchy of config scripts, some of which are from SVN, some from the UMUI output and some from override files. The basic structure of the output is given by container.cfg, which starts the include process. Variables are declared using a percent sign,
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
%variable %variable value
 
%variable %variable value
 
</syntaxhighlight>
 
</syntaxhighlight>
 
in bash syntax equates to
 
in bash syntax equates to
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
variable=$variable value
 
variable=$variable value
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 13: Line 13:
  
 
=Container=
 
=Container=
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/bindings/container.cfg | container.cfg]==  
+
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/bindings/container.cfg container.cfg]==
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
# Example parent file
 
# Example parent file
 
CFG::TYPE                        ext
 
CFG::TYPE                        ext
Line 53: Line 53:
 
The actual contents of the configuration are specified below, roughly in the order they're included into the final configuration (Some files are actually included in the middle of others)
 
The actual contents of the configuration are specified below, roughly in the order they're included into the final configuration (Some files are actually included in the middle of others)
  
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/machines/init_options.cfg | init_options.cfg]==  
+
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/machines/init_options.cfg init_options.cfg]==  
 
Declaration of variables
 
Declaration of variables
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
#######################################################################
 
#######################################################################
 
# Required libraries - libraries which are specified in the bindings. #
 
# Required libraries - libraries which are specified in the bindings. #
Line 116: Line 116:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/machines/linux-ifort-nci/machine.cfg | machine.cfg]==  
+
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/machines/linux-ifort-nci/machine.cfg machine.cfg]==  
 
Machine specific flags (This is for Vayu)
 
Machine specific flags (This is for Vayu)
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
# Defs related to building 64 bit executables
 
# Defs related to building 64 bit executables
 
%machdefs64 C_LONG_LONG_INT=c_long_long_int \
 
%machdefs64 C_LONG_LONG_INT=c_long_long_int \
Line 195: Line 195:
 
==FCM_UMUI_MODEL_CFG==  
 
==FCM_UMUI_MODEL_CFG==  
 
This is the FCM configuration produced by the UMUI. It adds flags for DrHook, Oasis &c when requested in the UI
 
This is the FCM configuration produced by the UMUI. It adds flags for DrHook, Oasis &c when requested in the UI
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
# File name: FCM_UMUI_MODEL_CFG
 
# File name: FCM_UMUI_MODEL_CFG
 
# Header:    Version number and user's logon
 
# Header:    Version number and user's logon
Line 344: Line 344:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/bindings/bind64_mpp_safe.cfg | bind64_mpp_safe.cfg]==  
+
==[https://access-svn.nci.org.au/trac/um/browser/branches/dev/Share/VN7.8_local_changes/src/configs/bindings/bind64_mpp_safe.cfg bind64_mpp_safe.cfg]==  
 
These are the actual compile flags to use, optimisation levels are added at this point
 
These are the actual compile flags to use, optimisation levels are added at this point
<syntaxhighlight>
+
<syntaxhighlight lang=text>
 
# Assign the 64 bit with MPP options defs, fflags and ldflags to the bld tools
 
# Assign the 64 bit with MPP options defs, fflags and ldflags to the bld tools
  

Latest revision as of 22:50, 11 December 2019

Template:Needs Update This page needs updating
Template:Pre UM10 This information is only for versions of the UM before 10.0

The FCM build script is created using a hierarchy of config scripts, some of which are from SVN, some from the UMUI output and some from override files. The basic structure of the output is given by container.cfg, which starts the include process. Variables are declared using a percent sign,

%variable %variable value

in bash syntax equates to

variable=$variable value

i.e. variables are expanded where they're declared (compare to Makefiles where variables expand where they are used).

Container

container.cfg

# Example parent file
CFG::TYPE                         ext
CFG::VERSION                      1.0

# Root dir of the build configuration
dest::rootdir    $UM_OUTDIR

# Remote location of build and config
rdest::machine      $UM_RHOST
rdest::rootdir      $UM_ROUTDIR
rdest::logname      $UM_RLOGNAME
rdest::remote_shell $UM_REMOTE_SHELL

# Standard UM base respository
repos::UM::base  $UM_SVN_URL/src/

# Source location in the repository
expsrc::UM::base

# Initialise some required options so FCM doesnt complain
inc $HERE/../machines/init_options.cfg

# Machine configuration file
inc $HERE/../machines/$UM_MACHINE/machine.cfg

# Include USER defined machine overrides
inc $UM_USR_MACH_OVRDS

# Job configuration file
inc $UM_JOB_CFG

# Include the user file overrides
inc $UM_USR_FILE_OVRDS

Contents

The actual contents of the configuration are specified below, roughly in the order they're included into the final configuration (Some files are actually included in the middle of others)

init_options.cfg

Declaration of variables

#######################################################################
# Required libraries - libraries which are specified in the bindings. #
#######################################################################

# GCOM library - to handle multi-processors
%fflags32_gcom_mpp
%fflags64_gcom_mpp
%ldflags32_gcom_mpp
%ldflags64_gcom_mpp

# GCOM serial library - serial version of the GCOM library.
%fflags32_gcom_serial
%fflags64_gcom_serial
%ldflags32_gcom_serial
%ldflags64_gcom_serial

# GRIB library - currently only used in reconfiguration and possibly to output
#                GRIB inside a PP file.
%fflags32_grib
%fflags64_grib
%ldflags32_grib
%ldflags64_grib

# Netcdf library - currently only used for SCM in the UM atmosphere model.
%fflags32_netcdf
%fflags64_netcdf
%ldflags32_netcdf
%ldflags64_netcdf

# Signal library - reports errors in calculations on IBM.
%fflags32_sig
%fflags64_sig
%ldflags32_sig
%ldflags64_sig

############################################################################
# Optional libraries - not strictly needed here but worth mentioning them.
#                      Included by the UMUI job if requested.
############################################################################

# Dr Hook library - reports diagnostic information useful for
#                   optimisation and debugging.
%fflags32_drhook
%fflags64_drhook
%ldflags32_drhook
%ldflags64_drhook

# OASIS3 library - coupler library only required from atmos-ocean models
#
%fflags64_oasis3
%ldflags64_oasis3

# Some flags which can be used and needed to be passed from
# machine.cfg or the UMUI to whatever bindings we use.
%cflags_opt
%mach_deps
%fflags_optional
%ldflags_optional

machine.cfg

Machine specific flags (This is for Vayu)

# Defs related to building 64 bit executables
%machdefs64 C_LONG_LONG_INT=c_long_long_int \
          C_LOW_U=c_low_u \
          FRL8=frl8 \
          LINUX=linux \
          BUFRD_IO=bufrd_io \
          LITTLE_END=little_end \
          LFS=lfs \
          _LARGEFILE_SOURCE=_largefile_source \
          _FILE_OFFSET_BITS=64

# Defs related to building 32 bit executables
%machdefs32 C_INT=c_int \
          C_LOW_U=c_low_u \
          LINUX=linux \
          BUFRD_IO=bufrd_io \
          LITTLE_END=little_end \
          LFS=lfs \
          _LARGEFILE_SOURCE=_largefile_source \
          _FILE_OFFSET_BITS=64

# The defs _LARGEFILE_SOURCE and _FILE_OFFSET_BITS are specific to gcc and
# enable largefile support in the compiler due to long type only being 32 bit.
# The defs should not be used in the UM.

# Standard names of the default bld::tools.
bld::tool::geninterface none
bld::tool::cpp      mpicc
bld::tool::cppflags -E
bld::tool::fppflags -E -P -traditional
bld::tool::ar       ar

%fc_serial          ifort
%fc_mpp             mpif90
%cc_serial          gcc
%cc_mpp             gcc
%ld_serial          ifort
%ld_mpp             mpif90

# Mandatory optimization flags
%fflags_opt_high  -O3 -g -traceback
%fflags_opt_safe  -O2 -g -traceback -fp-model precise -ftz
%fflags_opt_debug -O0 -g -traceback -fp-model precise -ftz

# Some internal variables
%ldflags_opt

# Mandatory exposed variables
%fflags64_serial        -g -i8 -r8
%fflags64_mpp           -g -i8 -r8

%ldflags64_serial       %ldflags_opt
%ldflags64_mpp          %ldflags_opt

%cflags_opt             -g -O3

# Some external libraries which have been defined as:
# fflags(32|64)_<lib>, ldflags(32|64)_<lib>

inc $HERE/ext_libs/default_paths.cfg

# If we have any paths to override then pick them up
# BEFORE we try to expand them.
inc $UM_USR_PATHS_OVRDS

inc $HERE/ext_libs/gcom_mpp.cfg
inc $HERE/ext_libs/gcom_serial.cfg
inc $HERE/ext_libs/netcdf.cfg
inc $HERE/ext_libs/totalview.cfg

Machine Overrides

Machine override files are included at this point

FCM_UMUI_MODEL_CFG

This is the FCM configuration produced by the UMUI. It adds flags for DrHook, Oasis &c when requested in the UI

# File name: FCM_UMUI_MODEL_CFG
# Header:    Version number and user's logon

version::UM::base    HEAD

# Part 1: Use Precompiled build or Base extract
#
# If UMUI switch "Use a precompiled build" is selected then the
# full path to the file ext.cfg in prebuild directory will be included,
# otherwise inherit src code and settings from base extract.
# For example:
#   use   $UM_PREBUILD/N48_atmos/ummodel/cfg/ext.cfg
# or
#   use   $UM_MAINDIR/umbase/cfg/ext.cfg

use $UM_MAINDIR/umbase/cfg/ext.cfg

# Part 2: Script build declarations (MANDATORY)
#

bld::excl_dep::UM::script  EXE
bld::excl_dep              USE::NetCDF
bld::excl_dep              INC::netcdf.inc
bld::excl_dep              INC::mpif.h
bld::excl_dep              USE::mpl
bld::excl_dep              USE::mod_prism_proto
bld::excl_dep              USE::mod_prism_grids_writing
bld::excl_dep              USE::mod_prism_def_partition_proto
bld::excl_dep              USE::mod_prism_put_proto
bld::excl_dep              USE::mod_prism_get_proto
bld::pp::UM                1

# Part 3a: Model specific sections (MANDATORY)

%jobdefs \
   CONTROL=control \
   REPROD=reprod \
   ATMOS=atmos \
   GLOBAL=global \
   A04_ALL=a04_all \
   A01_3C=a01_3c \
   A02_3C=a02_3c \
   A03_8C=a03_8c \
   A04_3D=a04_3d \
   A05_4A=a05_4a \
   A06_4A=a06_4a \
   A08_7A=a08_7a \
   A09_2A=a09_2a \
   A10_2A=a10_2a \
   A11_0A=a11_0a \
   A12_2A=a12_2a \
   A13_2A=a13_2a \
   A14_0A=a14_0a \
   A15_1A=a15_1a \
   A16_1A=a16_1a \
   A17_0A=a17_0a \
   A18_2A=a18_2a \
   A19_1A=a19_1a \
   A26_0A=a26_0a \
   A30_1A=a30_1a \
   A31_0A=a31_0a \
   A32_1A=a32_1a \
   A33_0A=a33_0a \
   A34_0A=a34_0a \
   A35_0A=a35_0a \
   A36_0A=a36_0a \
   A37_0A=a37_0a \
   A38_0A=a38_0a \
   A39_0A=a39_0a \
   A70_1C=a70_1c \
   A71_1A=a71_1a \
   C70_1A=c70_1a \
   C72_0A=c72_0a \
   C80_1A=c80_1a \
   C82_1A=c82_1a \
   C84_1A=c84_1a \
   C92_2A=c92_2a \
   C94_1A=c94_1a \
   C95_2A=c95_2a \
   C96_1C=c96_1c \
   C97_3A=c97_3a \
   C98_0A=c98_0a

# Part 3b: Name of executable, dependencies, target

bld::exe_name::flumeMain   $RUNID.exe
bld::exe_dep               portio2a.o pio_data_conv.o pio_io_timer.o %mach_deps
bld::target                $RUNID.exe
bld::blockdata             blkdata.o

# Part 3c: Optional flags

# Part 4: User overrides from a branch (OPTIONAL)
#
# If the user selects "Include modifications from branch(es)"
# then a pair of "branchN" declarations for each branch needs
# to be included.
# For example:
#   repos::UM::branch1    svn://fcm2/UM_svn/UM/branches/dev/hadbc/script_tests2/src
#   expsrc::UM::branch1   /

# Part 5: Overrides from central script mods (OPTIONAL)
#
# If the option "Use central script mods" in the umui is selected
# then "script_mods" declaration needs to be included. Really it
# is just another branch but with a predefined name.
# For example:
#   repos::UM::script_mods     svn://fcm2/UM_svn/UM/branches/dev/?????
#   expsrc::UM::script_mods    /

# Part 6: User overrides from a working copy (OPTIONAL)
#
# If the option "Include modifications from user working copy"
# is selected then "user" declarations needs to be included.
# For example:
#   repos::UM::user    /home/hc1100/hadbc/fcm_6.2/script_tests2/src
#   expsrc::UM::user

# Part 7: Pre-bindings UMUI controlled Libraries (OPTIONAL)
#
# The UMUI has options to include particular external
# libraries. Currently available libraries are:
#     DR HOOK
#     OASIS3

inc $UM_SVN_BIND/../machines/$UM_MACHINE/opt_libs/mach_ovr/drhook.cfg@HEAD

# Part 8: Bind file (MANDATORY)
#
# These relate to the level of optimisation required and sets up
# all the correct pre-processing, compiler and linker flags.
# For umui purposes there are 6 possibilities:
#   bind64_mpp_high.cfg
#   bind64_mpp_safe.cfg
#   bind64_mpp_debug.cfg
#   bind64_serial_high.cfg
#   bind64_serial_safe.cfg
#   bind64_serial_debug.cfg

inc $UM_SVN_BIND/bind64_mpp_safe.cfg@HEAD

# Part 9: Post-bindings UMUI controlled Libraries (OPTIONAL)
#
# The UMUI has options to include particular external
# libraries. Currently available libraries are:
#     OASIS3

bind64_mpp_safe.cfg

These are the actual compile flags to use, optimisation levels are added at this point

# Assign the 64 bit with MPP options defs, fflags and ldflags to the bld tools

%fflags64_extlibs %fflags64_gcom_mpp %fflags64_sig %fflags64_netcdf %fflags64_grib
%fflags64 %fflags64_mpp %fflags_opt_safe %fflags64_extlibs %fflags_optional
%ldflags64_extlibs %ldflags64_gcom_mpp %ldflags64_sig %ldflags64_netcdf %ldflags64_grib
%ldflags64 %ldflags64_mpp %ldflags64_extlibs %ldflags_optional

bld::tool::fc         %fc_mpp
bld::tool::cc         %cc_mpp
bld::tool::ld         %ld_mpp
bld::tool::fppkeys    %machdefs64 %jobdefs
bld::tool::cppkeys    %machdefs64 %jobdefs
bld::tool::fflags     %fflags64
bld::tool::cflags     %cflags_opt
bld::tool::ldflags    %ldflags64

inc $HERE/../machines/$UM_MACHINE/overrides64_safe.cfg

File Overrides

Any file-specific overrides specified in the UMUI are added after everything else has been specified