Difference between revisions of "News"

Line 1: Line 1:
  
==='''WRF v4.0.2'''===  
+
== '''2019-03-15''' ==
* This WRF version is now available on Raijin. Please refer to instructions [[WRF v4.0.2 installation | here]].
+
 
* The UNSW-CCRC-WRF code has also been updated to this version. Please see the V4.0.2 branch of [https://bitbucket.org/dargueso/unsw-ccrc-wrf/src/V4.0.2/ | the repository].
+
=== '''ERA5 update''' ===
=='''2018-10-24'''==  
+
 
==='''Python 2 end of support'''===  
+
*We are updating ERA5 back to 1979, some surface variables are already available for the entire period. Original published U and V files between 2000-2008 contained errors, we have downloaded the newly published corrected versions.<br/> More information on [[ERA5|ERA5]].
* Numpy has announced it will stop support for Python 2 from January 2019. This means numpy will still be available for Python 2 but the version for Python 2 will be frozen from then on. Following this announcement, several packages built upon numpy have announced the end of support for Python 2 as well. As such, we will stop updating our conda environments at NCI for Python 2 from January 2019 onwards. This means, the environments will still be available but no update will be made to them. We strongly advise all Python users to transition to Python 3.
+
 
===mppnccombine-fast===  
+
=== '''CMIP6 reminder''' ===
* If you are running MOM, you may need to collate your outputs or even your restarts. There is a new mppnccombine availble, faster than the old one. Aidan has mentioned during [https://www.youtube.com/watch?v=p1Sm_NGOuPc&t=257s | the training on payu new features ]and he put the information on using it in [https://climate-cms.org/mom,/collate,/mpi/2018/10/19/mppnccombine-fast.html | this blogpost].
+
 
=='''2018-09-25'''==  
+
*CMIP6 is managed by NCI in collaboration with the Centre, CSIRO and the BoM on some specific points. This means all official communication around CMIP6 is coming and will come from NCI. Do not ignore it if you are interested in CMIP6. Lastly, a link to the current, official [https://opus.nci.org.au/display/CMIP/CMIP+Community+Home wiki/website for CMIP6 in Australia]. Bookmark it!
==='''ERA5'''===  
+
 
* We know accept some requests for downloading ERA5 data. Please refer to [[ERA5 | the ERA5 wiki page]] for more information on the specific conditions.
+
=== '''WRF v4.0.2''' ===
==='''CMEMS-sea level v4.0'''===  
+
 
* There's a new version 4.0 available for CMEMS-sea level (previously known as AVISO). For more info see the relevant [[sealevel_GLO_PHY_L4_REP_observations_008_047 | wiki page]]
+
*This WRF version is now available on Raijin. Please refer to instructions [[WRF_v4.0.2_installation|here]].  
==='''JRA55 moving on Monday 13th August'''===  
+
*The UNSW-CCRC-WRF code has also been updated to this version. Please see the V4.0.2 branch of [https://bitbucket.org/dargueso/unsw-ccrc-wrf/src/V4.0.2/ the repository].  
* We are going to move JRA55 (not JRA55-do!!!) 6hr and day data, from
+
 
>> /g/data1/rr7/ana4MIPs/reanalysis/JRA/JRA55 to
+
== '''2018-10-24''' ==
>> /g/data1/rr7/JRA55
+
 
===CMS Trainings===  
+
=== '''Python 2 end of support''' ===
* Some trainings were recorded. You can find them on [https://www.youtube.com/channel/UCSmoK6oWV9O0Hmyt9UdDNsQ | the CMS YouTube channel]
+
 
* 3D Visualisation - Martin Jucker, UNSW https://youtu.be/ya-xXY5uzeo
+
*Numpy has announced it will stop support for Python 2 from January 2019. This means numpy will still be available for Python 2 but the version for Python 2 will be frozen from then on. Following this announcement, several packages built upon numpy have announced the end of support for Python 2 as well. As such, we will stop updating our conda environments at NCI for Python 2 from January 2019 onwards. This means, the environments will still be available but no update will be made to them. We strongly advise all Python users to transition to Python 3.  
* Nectar Research Cloud - Just Berkhout, TPAC https://youtu.be/8jluqwOOS0U
+
 
 +
=== mppnccombine-fast ===
 +
 
 +
*If you are running MOM, you may need to collate your outputs or even your restarts. There is a new mppnccombine availble, faster than the old one. Aidan has mentioned during [https://www.youtube.com/watch?v=p1Sm_NGOuPc&t=257s | the training on payu new features ]and he put the information on using it in [https://climate-cms.org/mom,/collate,/mpi/2018/10/19/mppnccombine-fast.html | this blogpost].  
 +
 
 +
== '''2018-09-25''' ==
 +
 
 +
=== '''ERA5''' ===
 +
 
 +
*We know accept some requests for downloading ERA5 data. Please refer to [[ERA5|the ERA5 wiki page]] for more information on the specific conditions.  
 +
 
 +
=== '''CMEMS-sea level v4.0''' ===
 +
 
 +
*There's a new version 4.0 available for CMEMS-sea level (previously known as AVISO). For more info see the relevant [[Sealevel_GLO_PHY_L4_REP_observations_008_047|wiki page]]  
 +
 
 +
=== '''JRA55 moving on Monday 13th August''' ===
 +
 
 +
*We are going to move JRA55 (not JRA55-do!!!) 6hr and day data, from  
 +
 
 +
>> /g/data1/rr7/ana4MIPs/reanalysis/JRA/JRA55 to >> /g/data1/rr7/JRA55
 +
 
 +
=== CMS Trainings ===
 +
 
 +
*Some trainings were recorded. You can find them on [https://www.youtube.com/channel/UCSmoK6oWV9O0Hmyt9UdDNsQ | the CMS YouTube channel]  
 +
*3D Visualisation - Martin Jucker, UNSW [https://youtu.be/ya-xXY5uzeo https://youtu.be/ya-xXY5uzeo]
 +
*Nectar Research Cloud - Just Berkhout, TPAC [https://youtu.be/8jluqwOOS0U https://youtu.be/8jluqwOOS0U]
 +
 
 +
== '''2018-08-01''' ==
 +
 
 +
=== '''New blog''' ===
 +
 
 +
*We are starting a blog to share answers to queries we receive. The aim is to help newcomers get started. The blog is at this address: [[http://climate-cms.org/ [1]]]
 +
 
 +
=== CMORPH moving ===
 +
 
 +
*We are going to move CMORPH to a different project, from ua8 to rr7, this is to group all re-analysis that do not have specific licensing terms under one project. The new location will be /g/data/rr7/CMORPH/. More details will follow.
 +
 
 +
=== Aerosol data from MACC reanalysis ===
 +
 
 +
*A small subset of the Monitoring Atmospheric Composition and Climate reanalysis from ECMWF is now available in the ub4 project. We downloaded only aerosols variables for the entire period (2003-2012). See [[MACC|MACC page]] for details
 +
 
 +
=== CMORPH update ===
 +
 
 +
*Since there are delays in releasing new data for V1-0, we downloaded its precursor V0.x from July 2017 to date included. We also concatenated the daily data into monthly files. See [[CMORPH|CMORPH page]] for more information.
 +
 
 +
=== Python package for selecting regions ===
 +
 
 +
*We came across [http://regionmask.readthedocs.io/en/stable/index.html | this package]. It is designed to help select geographical regions either predefined or defined by the users. Let us know if you would be interested in it and we'll get it installed in the conda package.
 +
 
 +
== 2018-05-14 ==
 +
 
 +
=== GLEAM v3.2 ===
 +
 
 +
&nbsp;
 +
 
 +
*updated versions of the GLEAM global datasets of terrestrial evaporation and root-zone soil moisture (GLEAM v3.2a and GLEAM v3.2b) are available in /g/data/wd9/BenchMarking/GLEAM_v3-2/
 +
 
 +
&nbsp;
 +
 
 +
=== UM performance with GA7 ===
 +
 
 +
*[[GA7_Performance|At this page]], you'll see a summary of the performance of the latest UM versions with GA7 configuration.
 +
 
 +
== 2018-03-28 ==
 +
 
 +
=== Publishing data on the NOAA/WDS Paleoclimatology archive ===
 +
 
 +
*We have started listing our published paleoclimate datasets on the NOAA/WDS Paleoclimatology archive:[https://www.ncdc.noaa.gov/paleo-search/study/23470 https://www.ncdc.noaa.gov/paleo-search/study/23470]
  
=='''2018-08-01'''==
 
==='''New blog'''===
 
* We are starting a blog to share answers to queries we receive. The aim is to help newcomers get started. The blog is at this address: [[http://climate-cms.org/]]
 
===CMORPH moving===
 
* We are going to move CMORPH to a different project, from ua8 to rr7, this is to group all re-analysis that do not have specific licensing terms under one project. The new location will be /g/data/rr7/CMORPH/. More details will follow.
 
===Aerosol data from MACC reanalysis===
 
* A small subset of the Monitoring Atmospheric Composition and Climate reanalysis from ECMWF is now available in the ub4 project. We downloaded only aerosols variables for the entire period (2003-2012). See [[MACC | MACC page]] for details
 
===CMORPH update===
 
* Since there are delays in releasing new data for V1-0, we downloaded its precursor V0.x from July 2017 to date included. We also concatenated the daily data into monthly files. See [[CMORPH | CMORPH page]] for more information.
 
===Python package for selecting regions===
 
* We came across [http://regionmask.readthedocs.io/en/stable/index.html | this package]. It is designed to help select geographical regions either predefined or defined by the users. Let us know if you would be interested in it and we'll get it installed in the conda package.
 
==2018-05-14==
 
===GLEAM v3.2===
 
<span style="background-color: #ffffff; display: block; font-family: -webkit-standard; font-size: medium; text-align: justify;">
 
* updated versions of the GLEAM global datasets of terrestrial evaporation and root-zone soil moisture (GLEAM v3.2a and GLEAM v3.2b) are available in /g/data/wd9/BenchMarking/GLEAM_v3-2/
 
</span>
 
===UM performance with GA7===
 
* [[GA7 Performance | At this page]], you'll see a summary of the performance of the latest UM versions with GA7 configuration.
 
==2018-03-28==
 
===Publishing data on the NOAA/WDS Paleoclimatology archive===
 
* We have started listing our published paleoclimate datasets on the NOAA/WDS Paleoclimatology archive:https://www.ncdc.noaa.gov/paleo-search/study/23470
 
 
> If you would like to do the same for your data or you know of another discipline specific data repository please contact us on climate-help
 
> If you would like to do the same for your data or you know of another discipline specific data repository please contact us on climate-help
===CESM - CCSM data===  
+
 
* We moved the CESM1-CAM5-BGC-LE, CESM1-LME and CCSM dataset to ua6, in line with their licensing terms. We also re-organised the data in a CMIP-like structure. Work is still going on, as well new variables have been added. We will update the wiki asap.
+
=== CESM - CCSM data ===
===CMIP5 - CORDEX - CNRM5 6hrLev recall===  
+
 
* There is an issue with CNRM5 historical 6hrLev, this can affect also CORDEX data for more information please refer to the [https://opus.nci.org.au/pages/viewpage.action?pageId=6324228 | CMIP confluence page]
+
*We moved the CESM1-CAM5-BGC-LE, CESM1-LME and CCSM dataset to ua6, in line with their licensing terms. We also re-organised the data in a CMIP-like structure. Work is still going on, as well new variables have been added. We will update the wiki asap.  
===GSWP3 data===  
+
 
* There is a new version for this dataset in the CABLE collection (wd9), the older version is considered obsolete now, it will be still available for a month in /g/data/ua8/GSWP3_obsolete and then deleted. We do not manage anymore this dataset so questions need to be directed to Bernard Pak (CSIRO).
+
=== CMIP5 - CORDEX - CNRM5 6hrLev recall ===
===OSTIA data===  
+
 
* We are stopping updates of this dataset since the users who requested originally do not need it anymore, if you are still using this data please contact us.
+
*There is an issue with CNRM5 historical 6hrLev, this can affect also CORDEX data for more information please refer to the [https://opus.nci.org.au/pages/viewpage.action?pageId=6324228 | CMIP confluence page]  
===Conda Updates===  
+
 
 +
=== GSWP3 data ===
 +
 
 +
*There is a new version for this dataset in the CABLE collection (wd9), the older version is considered obsolete now, it will be still available for a month in /g/data/ua8/GSWP3_obsolete and then deleted. We do not manage anymore this dataset so questions need to be directed to Bernard Pak (CSIRO).  
 +
 
 +
=== OSTIA data ===
 +
 
 +
*We are stopping updates of this dataset since the users who requested originally do not need it anymore, if you are still using this data please contact us.  
 +
 
 +
=== Conda Updates ===
 +
 
 
The Conda environments will be updated to a new version on Tuesday 27th March. This will be a general update of all packages, please email climate_help if you have any issues with python scripts after the update
 
The Conda environments will be updated to a new version on Tuesday 27th March. This will be a general update of all packages, please email climate_help if you have any issues with python scripts after the update
  
==2018-01-31==  
+
== 2018-01-31 ==
===WRF===  
+
 
* The Unified Post-Processing system (UPP) version 3.2 is now installed at NCI. It resides within the WRFV_3.9.1.1 directory but can be used with outputs from older WRF versions. For details on where to find it and how to compile it, refer to [[WRF#WRF-Post-processing%20software-UPP | this page]].
+
=== WRF ===
===Anaconda environment updates===  
+
 
CMS are changing the way we manage our Anaconda environments at NCI (Raijin, VDI and accessdev). To date we've been updating the environments every week, to improve stability of the environments we're going to be creating a 'stable' version
+
*The Unified Post-Processing system (UPP) version 3.2 is now installed at NCI. It resides within the WRFV_3.9.1.1 directory but can be used with outputs from older WRF versions. For details on where to find it and how to compile it, refer to [[WRF#WRF-Post-processing_software-UPP|this page]].  
of each environment which gets updated once per quarter.
+
 
 +
=== Anaconda environment updates ===
 +
 
 +
CMS are changing the way we manage our Anaconda environments at NCI (Raijin, VDI and accessdev). To date we've been updating the environments every week, to improve stability of the environments we're going to be creating a 'stable' version of each environment which gets updated once per quarter.
  
 
How you use the environments will stay the same, you can still use:
 
How you use the environments will stay the same, you can still use:
Line 72: Line 131:
  
 
As the number of packages being installed into our Anaconda environments is getting quite large reducing the update frequency will allow us to more thouroughly test the interactions between the different packages and ensure we've got a stable platform for research.
 
As the number of packages being installed into our Anaconda environments is getting quite large reducing the update frequency will allow us to more thouroughly test the interactions between the different packages and ensure we've got a stable platform for research.
===GPCP===
 
* Global Precipitation Climatology Project - daily precipitation analysis from surface and satellite measurements for 1997 onwards. Details in the [[GPCP | dataset page]] .
 
===GSMaP===
 
* <span style="font-size: 1.1em;">Global Satellite Mapping Precipitation re-analysis-gauge product is available in ua8. Details in the </span>[[GSMaP | dataset page]]
 
===MERRA-2 citation update===
 
* NASA has added DOIs and a citation for each of the MERRA-2 products. Details in the [[MERRA2 | dataset page]].
 
==2017-12-13==
 
==='''<span style="font-family: Arial; font-size: 11pt; vertical-align: baseline;">GIMMS AVHRR Global NDVI 1/12-degree</span>'''===
 
* Update: added files for 2011, 2012 and 2013. Data is in /g/data/ua8/gimmsndvi3g/<YYYYs_new>/<files>
 
===UM 10.6 GA7.1 n96 'HighResMIP'===
 
* The Hadley Centre's CMIP6 HighResMIP model can now be run on Raijin at the lower n96 resolution using the Rose job u-as709 - http://climate-cms.unsw.wikispaces.net/UM+Experiments
 
  
==2017-11-30==  
+
=== GPCP ===
===ERA5===  
+
 
* U,V and T operative analysis data for January 2016 is now available both as model level and pressure level (also Q) as grib and netcdf files. This data is provided as an example of ERA5. We'll be downloading also ENDA data for the same month. Data is in /g/data/ub4/era5-test/<format>/oper/<levels>/<files>
+
*Global Precipitation Climatology Project - daily precipitation analysis from surface and satellite measurements for 1997 onwards. Details in the [[GPCP|dataset page]] .
 +
 
 +
=== GSMaP ===
 +
 
 +
*<span style="font-size: 1.1em;">Global Satellite Mapping Precipitation re-analysis-gauge product is available in ua8. Details in the </span>[[GSMaP|dataset page]]
 +
 
 +
=== MERRA-2 citation update ===
 +
 
 +
*NASA has added DOIs and a citation for each of the MERRA-2 products. Details in the [[MERRA2|dataset page]].
 +
 
 +
== 2017-12-13 ==
 +
 
 +
=== '''<span style="font-family: Arial; font-size: 11pt; vertical-align: baseline;">GIMMS AVHRR Global NDVI 1/12-degree</span>''' ===
 +
 
 +
*Update: added files for 2011, 2012 and 2013. Data is in /g/data/ua8/gimmsndvi3g/<YYYYs_new>/<files>
 +
 
 +
=== UM 10.6 GA7.1 n96 'HighResMIP' ===
 +
 
 +
*The Hadley Centre's CMIP6 HighResMIP model can now be run on Raijin at the lower n96 resolution using the Rose job u-as709 - [http://climate-cms.unsw.wikispaces.net/UM+Experiments http://climate-cms.unsw.wikispaces.net/UM+Experiments]
 +
 
 +
== 2017-11-30 ==
 +
 
 +
=== ERA5 ===
 +
 
 +
*U,V and T operative analysis data for January 2016 is now available both as model level and pressure level (also Q) as grib and netcdf files. This data is provided as an example of ERA5. We'll be downloading also ENDA data for the same month. Data is in /g/data/ub4/era5-test/<format>/oper/<levels>/<files>  
 +
 
 +
=== New Wiki Pages ===
 +
 
 +
*We've added a guide to how to use the grib_api module to work with GRIB format files: [[GRIB_Format_Files|http://climate-cms.unsw.wikispaces.net/GRIB+Format+Files]]
 +
*Forcing files used for ACCESS CMIP5 runs: [[ACCESS_CMIP5_Forcings|http://climate-cms.unsw.wikispaces.net/ACCESS+CMIP5+Forcings]]
 +
*Information on NCI systems has been updated. We have also added some guidelines to using resources at NCI. [[NCI_Guidelines|http://climate-cms.unsw.wikispaces.net/NCI+Guidelines]]
 +
 
 +
=== Training material ===
 +
 
 +
*All the material from the training day can be found [https://drive.google.com/open?id=0B1KncyYETiZqNGdHZ0VZSk02cUE | here].
 +
 
 +
=== Unified Model Newsletter ===
 +
 
 +
*The latest UM newsletter is available at [https://code.metoffice.gov.uk/trac/jumps/raw-attachment/wiki/UMNewsletter/UMDec2017.pdf https://code.metoffice.gov.uk/trac/jumps/raw-attachment/wiki/UMNewsletter/UMDec2017.pdf]
 +
 
 +
== 2017-09-12 ==
 +
 
 +
=== WRF ===
 +
 
 +
*WRF v3.9.1.1 has now been ported to NCI. See [[WRF_v3.9.1.1_installation|here]].
 +
 
 +
=== ACCESS Science Day ===
 +
 
 +
*Claire and Scott presented talks on ESMValTool, ARCCSSive and higher resolution atmosphere models at the ACCESS Science Day, slides from all presentations are available at [https://accessdev.nci.org.au/trac/wiki/ScienceDay https://accessdev.nci.org.au/trac/wiki/ScienceDay]
 +
 
 +
=== CMIP5 clean up of extra copies ===
 +
 
 +
*As announced to anyone in ua6 by e-mail, we have initiated a clean up of extra copies of CMIP5 data. The process and a list of the redundant directories that eventually will be removed are available on the NCI [https://opus.nci.org.au/display/CMIP/2017/08/22/Notice+of+ua6+data+migration+and+clean+up | confluence site]
 +
 
 +
=== Updates to CMORPH data ===
 +
 
 +
*We checked the entire datasets for errors in files, we also updated the collection that now has data up to July 2017 included
 +
*The files for Sep 07 2003 and Jan 22 2014 were updated because they contained empty arrays.
 +
 
 +
=== Downloading from Met Office Archives (MASS/MOOSE) ===
 +
 
 +
*We are now able to download model outputs from the UK Met Office MASS archive system on request - [[Downloading_from_MASS|Downloading from MASS]]
 +
 
 +
=== Conda Updates ===
 +
 
 +
*===[https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda=== https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===]
 +
**[https://github.com/ioos/compliance-checker | compliance-checker] IOOS compliance checker for validating CF-NetCDF and ACDD metadata 
 +
*There were some issues with broken packages installed in the shared conda environment from August 31st to September 4th. We apologise for any inconvenience and are working on a robust testing solution to ensure this does not happen again in the future
 +
 
 +
== 2017-07-26 ==
 +
 
 +
=== NOAA OISST ===
 +
 
 +
*<span style="font-family: Arial,Helvetica,sans-serif;">NOAA is changing the delivery mode of OISST and also switching to netcdf4 format. I'm currently reviewing our download procedure and we'll finally have a new raw and modified version for this data. To help to transition the v2-0_modifed version will stay in place.</span>
 +
 
 +
=== WRF v3.9 ===
 +
 
 +
*The compiling scripts for WRF and WPS have been revamped. They now accept command-line arguments for customisation rather than requiring users to hack into the script. See details on the [[WRF_v3.9_installation|installation page for v3.9]]. If you would like more command-line options that's easy to add so please tell CMS about it.
 +
 
 +
=== PyPI publishing example ===
 +
 
 +
*A brief [[How_to_publish_your_Python_code_to_PyPI|HOW-TO on publishing simple code to PyPI]] which will make it more visible, and easier to install and use.
 +
 
 +
=== River runoff regridding tools ===
 +
 
 +
*[https://github.com/OceansAus/runoff_tools | River runoff regridding tools for MOM] written by Russ Fiedler for COSIMA have been installed as module on Raijin and VDI. See [[River_runoff_regridding_for_MOM|here]] for an example of how they are used.
 +
 
 +
=== JRA-55-do ===
 +
 
 +
*Repeat Year Forcing (RYF) derivative datasets of the Japanese Reanalysis for Ocean Model forcing (JRA55-do) are available under the same directory as JRA55-do. In addition there are regridded runoff data for different MOM ocean configurations available in ua8. Contact climate_help@nci.org.au for more information.
 +
 
 +
=== Conda Updates ===
 +
 
 +
*===[https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda=== https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===]
 +
**[https://code.metoffice.gov.uk/doc/um/mule/2017.06.1 | mule] The UKMO library for reading and writing UM format files
 +
**[https://www.ffmpeg.org | ffmpeg] Video and audio encoding library and utilities. Installed to support creating animations in matplotlib
 +
**[https://github.com/pypa/twine | twine] Utility to support installing PyPI packages
 +
**[https://github.com/pydap/pydap | pydap] Pydap is an implementation of the Opendap/DODS protocol 
 +
 
 +
=== STASH and Rose ===
 +
 
 +
*How to copy STASH settings between suites with Rose - [http://climate-cms.unsw.wikispaces.net/Running+the+UM+with+Rose#x-Copying%20STASH%20fields http://climate-cms.unsw.wikispaces.net/Running+the+UM+with+Rose#x-Copying%20STASH%20fields]
 +
 
 +
== 2017-06-06 ==
 +
 
 +
=== ERA-Interim for WRF ===
 +
 
 +
*Bug information: The information on how to use the ERA-Interim dataset in WPS on the wiki ([[How_to_run_WRF_with_ERA-Interim_data|here]]) and the automatic script [https://github.com/coecms/wps-era | wps-era] referenced on that page were erroneous. This code and instructions would potentially create erroneous fields along the coast lines. These instructions and script have now been updated. <span class="s1">Please update your code and regenerate your WRF input files if you have used wps-era</span>
 +
 
 +
=== TRMM ===
 +
 
 +
*Bug information: some files in the TRMM dataset at NCI got corrupted. The dataset has now been repaired and more stringent tests have been added to help avoid corrupted files in the future.
 +
 
 +
=== ACCESS-MOSES with KPP ===
 +
 
 +
*A version of ACCESS1.0 coupled to KPP using OASIS3-MCT was finalised. UMUI jobs <span style="font-family:monospace">vatae}} (N96 resolution) and {{vaski</span> (N48) can be used as a basis. Further documentation in [[ACCESS-MOSES-KPP|ACCESS-MOSES-KPP]].
 +
 
 +
== 2017-05-19 ==
 +
 
 +
=== CMIP induction and ARCCSSive training ===
 +
 
 +
*We added to the NCI online training website a CMIP induction which includes training on how to use the ARCCSSive python module. To access the course use your NCI account to login [https://training.nci.org.au/course/view.php?id=26 | here]
 +
 
 +
=== JRA55-do ===
 +
 
 +
*JRA-55 based surface dataset for driving ocean-sea ice models, JRA55-do v 0.8 and v1.1 are now available in /g/data1/ua8/JRA55-do/v0-8 (v1-1). This dataset is covered by a license and it's not officially published yet, so you shouldn't re-distribute it. Please, follow Terms of Use available in the main directory.
 +
 
 +
=== WRF v3.9 ===
 +
 
 +
*The latest version of WRF (v3.9) is now available on Raijin. Please find detailed instructions [[WRF_v3.9_installation|here]]
 +
 
 +
=== Conda Updates ===
 +
 
 +
*===[https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda=== https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===]
 +
**[http://wrf-python.readthedocs.io/en/latest/index.html | wrf-python] A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model.
 +
**[http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret | pyferret] PyFerret is a Python module wrapping Ferret. The pyferret module provides Python functions so Python users can easily take advantage of the Ferret's abilities to retrieve, manipulate, visualize, and save data. (Python 2.7 only)
 +
**[http://www.sympy.org/en/index.html | SymPy] SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python. 
 +
 
 +
== 2017-04-05 ==
 +
 
 +
=== Data collections Updates ===
 +
 
 +
*Expansion of MERRA-2 collection at NCI with the inst3d_3d_asm_Np data. [[MERRA2|View MERRA-2 page]]
 +
 
 +
=== UM10.6 GA7 ===
 +
 
 +
*The UK Met Office's Global Atmosphere 7 configuration is available on Raijin for UM 10.6 as job '<span style="font-family:monospace">u-ak072</span>' [http://climate-cms.unsw.wikispaces.net/GlobalAtmosphere http://climate-cms.unsw.wikispaces.net/GlobalAtmosphere]
 +
 
 +
=== LROSE ===
 +
 
 +
*NCAR's [https://www.eol.ucar.edu/content/lidar-radar-open-software-environment | Lidar Radar Open Software Environment (LROSE)] has been installed as a public module in <span style="font-family:monospace">/g/data3/hh5/public/modules. <span style="font-family: Arial,Helvetica,sans-serif;">The commands to load the module are:</span></span>
 +
 
 +
module use /g/data3/hh5/public/modules module load lrose
  
===New Wiki Pages===  
+
=== Conda Updates ===
* We've added a guide to how to use the grib_api module to work with GRIB format files: [[GRIB Format Files  |  http://climate-cms.unsw.wikispaces.net/GRIB+Format+Files]]
 
* Forcing files used for ACCESS CMIP5 runs: [[ACCESS CMIP5 Forcings  |  http://climate-cms.unsw.wikispaces.net/ACCESS+CMIP5+Forcings]]
 
* Information on NCI systems has been updated. We have also added some guidelines to using resources at NCI. [[NCI Guidelines  |  http://climate-cms.unsw.wikispaces.net/NCI+Guidelines]]
 
  
===Training material===  
+
*===Several packages have been added to the public Anaconda environments [https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda=== https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===]
* All the material from the training day can be found [https://drive.google.com/open?id=0B1KncyYETiZqNGdHZ0VZSk02cUE | here].
+
**[https://unidata.github.io/MetPy/ | metpy]: MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.
 +
**[http://ajdawson.github.io/windspharm/ | windspharm]: windspharm is a Python package for performing computations on global wind fields in spherical geometry.
 +
**[https://code.zmaw.de/projects/cdo/wiki/Cdo{rbpy} | python-cdo]: Python bindings for CDO
 +
**[http://arm-doe.github.io/pyart/ | pyart]: The Python ARM Radar Toolkit
 +
**[http://bokeh.pydata.org/en/latest/ | bokeh]: Bokeh is a Python interactive visualization library that targets modern web browsers for presentation.  
  
===Unified Model Newsletter===
+
== 2017-03-17 ==
* The latest UM newsletter is available at https://code.metoffice.gov.uk/trac/jumps/raw-attachment/wiki/UMNewsletter/UMDec2017.pdf
 
  
==2017-09-12==
+
=== Python libraries at NCI ===
===WRF===
 
* WRF v3.9.1.1 has now been ported to NCI. See [[WRF v3.9.1.1 installation | here]].
 
===ACCESS Science Day===  
 
* Claire and Scott presented talks on ESMValTool, ARCCSSive and higher resolution atmosphere models at the ACCESS Science Day, slides from all presentations are available at https://accessdev.nci.org.au/trac/wiki/ScienceDay
 
  
===CMIP5 clean up of extra copies===
+
*We've moved the Python libraries installed with Miniconda to a public space (/g/data3/hh5/public/modules) so there is no need anymore to be part of ACCESS to use it. See the [https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda | updated instructions]  
* As announced to anyone in ua6 by e-mail, we have initiated a clean up of extra copies of CMIP5 data. The process and a list of the redundant directories that eventually will be removed are available on the NCI [https://opus.nci.org.au/display/CMIP/2017/08/22/Notice+of+ua6+data+migration+and+clean+up | confluence site]
 
===Updates to CMORPH data===
 
* We checked the entire datasets for errors in files, we also updated the collection that now has data up to July 2017 included
 
* The files for Sep 07 2003 and Jan 22 2014 were updated because they contained empty arrays.
 
===Downloading from Met Office Archives (MASS/MOOSE)===
 
* We are now able to download model outputs from the UK Met Office MASS archive system on request - [[Downloading from MASS]]
 
  
===Conda Updates===
+
=== UM Newsletter ===
* ===https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
 
** [https://github.com/ioos/compliance-checker | compliance-checker] IOOS compliance checker for validating CF-NetCDF and ACDD metadata
 
* There were some issues with broken packages installed in the shared conda environment from August 31st to September 4th. We apologise for any inconvenience and are working on a robust testing solution to ensure this does not happen again in the future
 
==2017-07-26==
 
===NOAA OISST===
 
* <span style="font-family: Arial,Helvetica,sans-serif;">NOAA is changing the delivery mode of OISST and also switching to netcdf4 format. I'm currently reviewing our download procedure and we'll finally have a new raw and modified version for this data. To help to transition the v2-0_modifed version will stay in place.</span>
 
===WRF v3.9===  
 
* The compiling scripts for WRF and WPS have been revamped. They now accept command-line arguments for customisation rather than requiring users to hack into the script. See details on the [[WRF v3.9 installation | installation page for v3.9]]. If you would like more command-line options that's easy to add so please tell CMS about it.
 
===PyPI publishing example===
 
* A brief [[How to publish your Python code to PyPI | HOW-TO on publishing simple code to PyPI]] which will make it more visible, and easier to install and use.
 
===River runoff regridding tools===
 
* [https://github.com/OceansAus/runoff_tools | River runoff regridding tools for MOM] written by Russ Fiedler for COSIMA have been installed as module on Raijin and VDI. See [[River runoff regridding for MOM | here]] for an example of how they are used.
 
===JRA-55-do===
 
* Repeat Year Forcing (RYF) derivative datasets of the Japanese Reanalysis for Ocean Model forcing (JRA55-do) are available under the same directory as JRA55-do. In addition there are regridded runoff data for different MOM ocean configurations available in ua8. Contact climate_help@nci.org.au for more information.
 
===Conda Updates===
 
* ===https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
 
** [https://code.metoffice.gov.uk/doc/um/mule/2017.06.1 | mule] The UKMO library for reading and writing UM format files
 
** [https://www.ffmpeg.org | ffmpeg] Video and audio encoding library and utilities. Installed to support creating animations in matplotlib
 
** [https://github.com/pypa/twine | twine] Utility to support installing PyPI packages
 
** [https://github.com/pydap/pydap | pydap] Pydap is an implementation of the Opendap/DODS protocol
 
  
===STASH and Rose===
+
*The latest newsletter is released: <span class="s1">[http://collab.metoffice.gov.uk/twiki/bin/view/Support/UMNewsletterFeb2017 http://collab.metoffice.gov.uk/twiki/bin/view/Support/UMNewsletterFeb2017]</span>
* How to copy STASH settings between suites with Rose - http://climate-cms.unsw.wikispaces.net/Running+the+UM+with+Rose#x-Copying%20STASH%20fields
 
  
==2017-06-06==
+
=== <span class="s1">Massdata utility</span> ===
===ERA-Interim for WRF===
 
* Bug information: The information on how to use the ERA-Interim dataset in WPS on the wiki ([[How to run WRF with ERA-Interim data | here]]) and the automatic script [https://github.com/coecms/wps-era | wps-era] referenced on that page were erroneous. This code and instructions would potentially create erroneous fields along the coast lines. These instructions and script have now been updated. <span class="s1">Please update your code and regenerate your WRF input files if you have used wps-era</span>
 
===TRMM===
 
* Bug information: some files in the TRMM dataset at NCI got corrupted. The dataset has now been repaired and more stringent tests have been added to help avoid corrupted files in the future.
 
===ACCESS-MOSES with KPP===  
 
* A version of ACCESS1.0 coupled to KPP using OASIS3-MCT was finalised. UMUI jobs <span style="font-family:monospace">vatae}} (N96 resolution) and {{vaski</span> (N48) can be used as a basis. Further documentation in [[ACCESS-MOSES-KPP]].
 
  
==2017-05-19==
+
*<span class="s1">Are you moving data to/from massdata? Are you having issues with the copy stopping in the middle of things and impossibility to restart from where it failed? Aidan wrote a small utility to help with that. It’s available to anyone, info: <span class="s2">[[https://github.com/coecms/mdssdiff [2]]]</span></span>
===CMIP induction and ARCCSSive training===
 
* We added to the NCI online training website a CMIP induction which includes training on how to use the ARCCSSive python module. To access the course use your NCI account to login [https://training.nci.org.au/course/view.php?id=26 | here]
 
  
===JRA55-do===  
+
== 2017-02-15 ==
  
* JRA-55 based surface dataset for driving ocean-sea ice models, JRA55-do v 0.8 and v1.1 are now available in /g/data1/ua8/JRA55-do/v0-8 (v1-1). This dataset is covered by a license and it's not officially published yet, so you shouldn't re-distribute it. Please, follow Terms of Use available in the main directory.
+
=== MPAS-Atmosphere ===
  
===WRF v3.9===
+
*The latest release of [[MPAS|MPAS-Atmosphere]] has been installed on Raijin. This model will probably supplant WRF sometime in the future.  
* The latest version of WRF (v3.9) is now available on Raijin. Please find detailed instructions [[WRF v3.9 installation | here]]
 
  
===Conda Updates===  
+
=== Python libraries at NCI ===
* ===https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
 
** [http://wrf-python.readthedocs.io/en/latest/index.html | wrf-python] A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model.
 
** [http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret | pyferret] PyFerret is a Python module wrapping Ferret. The pyferret module provides Python functions so Python users can easily take advantage of the Ferret's abilities to retrieve, manipulate, visualize, and save data. (Python 2.7 only)
 
** [http://www.sympy.org/en/index.html | SymPy] SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python.
 
  
==2017-04-05==
+
*We've installed some Python libraries using Miniconda in /g/data1/access. The aim is to facilitate access to and management of commonly used Python libraries on Raijin, Accessdev and VDI. It also enables users to locally install their own libraries using Miniconda without duplicating the centrally installed libraries. For the moment, users have to be part of the ACCESS group on Raijin to use these (if you are not, just ask access on my.nci.org.au) but we plan to make it public access. More details are [https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda | here].
===Data collections Updates===
 
* Expansion of MERRA-2 collection at NCI with the inst3d_3d_asm_Np data. [[MERRA2 | View MERRA-2 page]]
 
  
===UM10.6 GA7===  
+
=== Sharing facility at NCI ===
* The UK Met Office's Global Atmosphere 7 configuration is available on Raijin for UM 10.6 as job '<span style="font-family:monospace">u-ak072</span>' http://climate-cms.unsw.wikispaces.net/GlobalAtmosphere
 
  
===LROSE===
+
*We worked at NCI to enable users to share data stored on Raijn with collaborators who don't have access to NCI. This is aimed for work in progress which can not be published and opened publicly. The files are accessible for a short time only to allow your collaborators to download them to their own storage. We are working on making it user-friendly but if you need to share some data in the meantime, please ask climate_help@nci.org.au.
* NCAR's [https://www.eol.ucar.edu/content/lidar-radar-open-software-environment | Lidar Radar Open Software Environment (LROSE)] has been installed as a public module in <span style="font-family:monospace">/g/data3/hh5/public/modules. <span style="font-family: Arial,Helvetica,sans-serif;">The commands to load the module are:</span></span>
 
module use /g/data3/hh5/public/modules
 
module load lrose
 
  
===Conda Updates===  
+
=== Updating OpenMPI library for MOM ===
* ===Several packages have been added to the public Anaconda environments https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
 
** [https://unidata.github.io/MetPy/ | metpy]: MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.
 
** [http://ajdawson.github.io/windspharm/ | windspharm]: windspharm is a Python package for performing computations on global wind fields in spherical geometry.
 
** [https://code.zmaw.de/projects/cdo/wiki/Cdo%7Brbpy%7D | python-cdo]: Python bindings for CDO
 
** [http://arm-doe.github.io/pyart/ | pyart]: The Python ARM Radar Toolkit
 
** [http://bokeh.pydata.org/en/latest/ | bokeh]: Bokeh is a Python interactive visualization library that targets modern web browsers for presentation.
 
  
==2017-03-17==
+
*After the new quarter maintenance, some MOM jobs experienced some stalling behaviour at the end of the job. This mainly affect large configurations (0.1°). The fix is to recompile MOM using the openmpi/10.2.1 version. If you don't know whether you should worry about it or how to recompile, please contact climate_help@nci.org.au.  
===Python libraries at NCI===
 
* We've moved the Python libraries installed with Miniconda to a public space (/g/data3/hh5/public/modules) so there is no need anymore to be part of ACCESS to use it. See the [https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda | updated instructions]
 
===UM Newsletter===
 
* The latest newsletter is released: <span class="s1">http://collab.metoffice.gov.uk/twiki/bin/view/Support/UMNewsletterFeb2017</span>
 
===<span class="s1">Massdata utility</span>===
 
* <span class="s1">Are you moving data to/from massdata? Are you having issues with the copy stopping in the middle of things and impossibility to restart from where it failed? Aidan wrote a small utility to help with that. It’s available to anyone, info: <span class="s2">[[https://github.com/coecms/mdssdiff]]</span></span>
 
==2017-02-15==
 
  
===MPAS-Atmosphere===  
+
=== NU-WRF ===
* The latest release of [[MPAS | MPAS-Atmosphere]] has been installed on Raijin. This model will probably supplant WRF sometime in the future.
 
  
===Python libraries at NCI===
+
*The latest manual is now on the CMS wiki. Just follow the NU-WRF link in the navigation sidebar.  
* We've installed some Python libraries using Miniconda in /g/data1/access. The aim is to facilitate access to and management of commonly used Python libraries on Raijin, Accessdev and VDI. It also enables users to locally install their own libraries using Miniconda without duplicating the centrally installed libraries. For the moment, users have to be part of the ACCESS group on Raijin to use these (if you are not, just ask access on my.nci.org.au) but we plan to make it public access. More details are [https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda | here].
 
  
===Sharing facility at NCI===
+
== 2016-08-22 ==
* We worked at NCI to enable users to share data stored on Raijn with collaborators who don't have access to NCI. This is aimed for work in progress which can not be published and opened publicly. The files are accessible for a short time only to allow your collaborators to download them to their own storage. We are working on making it user-friendly but if you need to share some data in the meantime, please ask climate_help@nci.org.au.
 
  
===Updating OpenMPI library for MOM===  
+
=== Configurations ===
* After the new quarter maintenance, some MOM jobs experienced some stalling behaviour at the end of the job. This mainly affect large configurations (0.1°). The fix is to recompile MOM using the openmpi/10.2.1 version. If you don't know whether you should worry about it or how to recompile, please contact climate_help@nci.org.au.
 
  
===NU-WRF===
+
*ACCESS CM 1.0 pre-industrial control has been ported to work on Raijin [https://accessdev.nci.org.au/trac/wiki/access/ACCESS1.0_raijin#piControl https://accessdev.nci.org.au/trac/wiki/access/ACCESS1.0_raijin#piControl]
* The latest manual is now on the CMS wiki. Just follow the NU-WRF link in the navigation sidebar.
 
  
==2016-08-22<span id="2016-08-22"></span>==  
+
=== UM ===
  
===Configurations===
+
*The Accesscollab UMUI server is no longer supported, and we can't guarantee the safety of data on it. Please move any data on the 'Collab' page of the UMUI to the 'Accessdev' or 'Collab(local)' page. The 'Collab (local)' page contains copies of the Accescollab database from last year, if your jobs are present there no further action is required.  
* ACCESS CM 1.0 pre-industrial control has been ported to work on Raijin https://accessdev.nci.org.au/trac/wiki/access/ACCESS1.0_raijin#piControl
 
  
===UM===
 
* The Accesscollab UMUI server is no longer supported, and we can't guarantee the safety of data on it. Please move any data on the 'Collab' page of the UMUI to the 'Accessdev' or 'Collab(local)' page. The 'Collab (local)' page contains copies of the Accescollab database from last year, if your jobs are present there no further action is required.
 
 
> To move an experiment folder right click on it and select 'copy' then go to the 'accessedev' or 'collab(local)' page and use the 'paste' button in the top right of umuix to copy all jobs in that folder.
 
> To move an experiment folder right click on it and select 'copy' then go to the 'accessedev' or 'collab(local)' page and use the 'paste' button in the top right of umuix to copy all jobs in that folder.
* NIWA's um2netcdf.x conversion script is now available as ACCESS module <span style="font-family:monospace">um2netcdf-niwa</span> https://github.com/adeytown/um2netcdf too
+
 
 +
*NIWA's um2netcdf.x conversion script is now available as ACCESS module <span style="font-family:monospace">um2netcdf-niwa</span> [https://github.com/adeytown/um2netcdf https://github.com/adeytown/um2netcdf] too

Revision as of 21:24, 14 March 2019

Contents

2019-03-15

ERA5 update

  • We are updating ERA5 back to 1979, some surface variables are already available for the entire period. Original published U and V files between 2000-2008 contained errors, we have downloaded the newly published corrected versions.
    More information on ERA5.

CMIP6 reminder

  • CMIP6 is managed by NCI in collaboration with the Centre, CSIRO and the BoM on some specific points. This means all official communication around CMIP6 is coming and will come from NCI. Do not ignore it if you are interested in CMIP6. Lastly, a link to the current, official wiki/website for CMIP6 in Australia. Bookmark it!

WRF v4.0.2

  • This WRF version is now available on Raijin. Please refer to instructions here.
  • The UNSW-CCRC-WRF code has also been updated to this version. Please see the V4.0.2 branch of the repository.

2018-10-24

Python 2 end of support

  • Numpy has announced it will stop support for Python 2 from January 2019. This means numpy will still be available for Python 2 but the version for Python 2 will be frozen from then on. Following this announcement, several packages built upon numpy have announced the end of support for Python 2 as well. As such, we will stop updating our conda environments at NCI for Python 2 from January 2019 onwards. This means, the environments will still be available but no update will be made to them. We strongly advise all Python users to transition to Python 3.

mppnccombine-fast

  • If you are running MOM, you may need to collate your outputs or even your restarts. There is a new mppnccombine availble, faster than the old one. Aidan has mentioned during | the training on payu new features and he put the information on using it in | this blogpost.

2018-09-25

ERA5

  • We know accept some requests for downloading ERA5 data. Please refer to the ERA5 wiki page for more information on the specific conditions.

CMEMS-sea level v4.0

  • There's a new version 4.0 available for CMEMS-sea level (previously known as AVISO). For more info see the relevant wiki page

JRA55 moving on Monday 13th August

  • We are going to move JRA55 (not JRA55-do!!!) 6hr and day data, from

>> /g/data1/rr7/ana4MIPs/reanalysis/JRA/JRA55 to >> /g/data1/rr7/JRA55

CMS Trainings

2018-08-01

New blog

  • We are starting a blog to share answers to queries we receive. The aim is to help newcomers get started. The blog is at this address: [[1]]

CMORPH moving

  • We are going to move CMORPH to a different project, from ua8 to rr7, this is to group all re-analysis that do not have specific licensing terms under one project. The new location will be /g/data/rr7/CMORPH/. More details will follow.

Aerosol data from MACC reanalysis

  • A small subset of the Monitoring Atmospheric Composition and Climate reanalysis from ECMWF is now available in the ub4 project. We downloaded only aerosols variables for the entire period (2003-2012). See MACC page for details

CMORPH update

  • Since there are delays in releasing new data for V1-0, we downloaded its precursor V0.x from July 2017 to date included. We also concatenated the daily data into monthly files. See CMORPH page for more information.

Python package for selecting regions

  • We came across | this package. It is designed to help select geographical regions either predefined or defined by the users. Let us know if you would be interested in it and we'll get it installed in the conda package.

2018-05-14

GLEAM v3.2

 

  • updated versions of the GLEAM global datasets of terrestrial evaporation and root-zone soil moisture (GLEAM v3.2a and GLEAM v3.2b) are available in /g/data/wd9/BenchMarking/GLEAM_v3-2/

 

UM performance with GA7

  • At this page, you'll see a summary of the performance of the latest UM versions with GA7 configuration.

2018-03-28

Publishing data on the NOAA/WDS Paleoclimatology archive

> If you would like to do the same for your data or you know of another discipline specific data repository please contact us on climate-help

CESM - CCSM data

  • We moved the CESM1-CAM5-BGC-LE, CESM1-LME and CCSM dataset to ua6, in line with their licensing terms. We also re-organised the data in a CMIP-like structure. Work is still going on, as well new variables have been added. We will update the wiki asap.

CMIP5 - CORDEX - CNRM5 6hrLev recall

  • There is an issue with CNRM5 historical 6hrLev, this can affect also CORDEX data for more information please refer to the | CMIP confluence page

GSWP3 data

  • There is a new version for this dataset in the CABLE collection (wd9), the older version is considered obsolete now, it will be still available for a month in /g/data/ua8/GSWP3_obsolete and then deleted. We do not manage anymore this dataset so questions need to be directed to Bernard Pak (CSIRO).

OSTIA data

  • We are stopping updates of this dataset since the users who requested originally do not need it anymore, if you are still using this data please contact us.

Conda Updates

The Conda environments will be updated to a new version on Tuesday 27th March. This will be a general update of all packages, please email climate_help if you have any issues with python scripts after the update

2018-01-31

WRF

  • The Unified Post-Processing system (UPP) version 3.2 is now installed at NCI. It resides within the WRFV_3.9.1.1 directory but can be used with outputs from older WRF versions. For details on where to find it and how to compile it, refer to this page.

Anaconda environment updates

CMS are changing the way we manage our Anaconda environments at NCI (Raijin, VDI and accessdev). To date we've been updating the environments every week, to improve stability of the environments we're going to be creating a 'stable' version of each environment which gets updated once per quarter.

How you use the environments will stay the same, you can still use:

   module use /g/data3/hh5/public/modules
   module load conda/analysis3 # or conda/analysis27

and get access to a wide variety of climate and weather focused Python libraries.

We'll still be installing new packages on request, you can ask for them by emailing us at climate_help@nci.org.au, but so that the 'stable' environments are not affected new packages will be installed into the environments 'analysis3-unstable' and 'analysis27-unstable'. During the quarterly updates all of the new packages added during the past quarter will be added to the 'stable' environments.

As the number of packages being installed into our Anaconda environments is getting quite large reducing the update frequency will allow us to more thouroughly test the interactions between the different packages and ensure we've got a stable platform for research.

GPCP

  • Global Precipitation Climatology Project - daily precipitation analysis from surface and satellite measurements for 1997 onwards. Details in the dataset page .

GSMaP

  • Global Satellite Mapping Precipitation re-analysis-gauge product is available in ua8. Details in the dataset page

MERRA-2 citation update

  • NASA has added DOIs and a citation for each of the MERRA-2 products. Details in the dataset page.

2017-12-13

GIMMS AVHRR Global NDVI 1/12-degree

  • Update: added files for 2011, 2012 and 2013. Data is in /g/data/ua8/gimmsndvi3g/<YYYYs_new>/<files>

UM 10.6 GA7.1 n96 'HighResMIP'

2017-11-30

ERA5

  • U,V and T operative analysis data for January 2016 is now available both as model level and pressure level (also Q) as grib and netcdf files. This data is provided as an example of ERA5. We'll be downloading also ENDA data for the same month. Data is in /g/data/ub4/era5-test/<format>/oper/<levels>/<files>

New Wiki Pages

Training material

  • All the material from the training day can be found | here.

Unified Model Newsletter

2017-09-12

WRF

  • WRF v3.9.1.1 has now been ported to NCI. See here.

ACCESS Science Day

CMIP5 clean up of extra copies

  • As announced to anyone in ua6 by e-mail, we have initiated a clean up of extra copies of CMIP5 data. The process and a list of the redundant directories that eventually will be removed are available on the NCI | confluence site

Updates to CMORPH data

  • We checked the entire datasets for errors in files, we also updated the collection that now has data up to July 2017 included
  • The files for Sep 07 2003 and Jan 22 2014 were updated because they contained empty arrays.

Downloading from Met Office Archives (MASS/MOOSE)

  • We are now able to download model outputs from the UK Met Office MASS archive system on request - Downloading from MASS

Conda Updates

2017-07-26

NOAA OISST

  • NOAA is changing the delivery mode of OISST and also switching to netcdf4 format. I'm currently reviewing our download procedure and we'll finally have a new raw and modified version for this data. To help to transition the v2-0_modifed version will stay in place.

WRF v3.9

  • The compiling scripts for WRF and WPS have been revamped. They now accept command-line arguments for customisation rather than requiring users to hack into the script. See details on the installation page for v3.9. If you would like more command-line options that's easy to add so please tell CMS about it.

PyPI publishing example

River runoff regridding tools

JRA-55-do

  • Repeat Year Forcing (RYF) derivative datasets of the Japanese Reanalysis for Ocean Model forcing (JRA55-do) are available under the same directory as JRA55-do. In addition there are regridded runoff data for different MOM ocean configurations available in ua8. Contact climate_help@nci.org.au for more information.

Conda Updates

STASH and Rose

2017-06-06

ERA-Interim for WRF

  • Bug information: The information on how to use the ERA-Interim dataset in WPS on the wiki (here) and the automatic script | wps-era referenced on that page were erroneous. This code and instructions would potentially create erroneous fields along the coast lines. These instructions and script have now been updated. Please update your code and regenerate your WRF input files if you have used wps-era

TRMM

  • Bug information: some files in the TRMM dataset at NCI got corrupted. The dataset has now been repaired and more stringent tests have been added to help avoid corrupted files in the future.

ACCESS-MOSES with KPP

  • A version of ACCESS1.0 coupled to KPP using OASIS3-MCT was finalised. UMUI jobs vatae}} (N96 resolution) and {{vaski (N48) can be used as a basis. Further documentation in ACCESS-MOSES-KPP.

2017-05-19

CMIP induction and ARCCSSive training

  • We added to the NCI online training website a CMIP induction which includes training on how to use the ARCCSSive python module. To access the course use your NCI account to login | here

JRA55-do

  • JRA-55 based surface dataset for driving ocean-sea ice models, JRA55-do v 0.8 and v1.1 are now available in /g/data1/ua8/JRA55-do/v0-8 (v1-1). This dataset is covered by a license and it's not officially published yet, so you shouldn't re-distribute it. Please, follow Terms of Use available in the main directory.

WRF v3.9

  • The latest version of WRF (v3.9) is now available on Raijin. Please find detailed instructions here

Conda Updates

  • ===https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
    • | wrf-python A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model.
    • | pyferret PyFerret is a Python module wrapping Ferret. The pyferret module provides Python functions so Python users can easily take advantage of the Ferret's abilities to retrieve, manipulate, visualize, and save data. (Python 2.7 only)
    • | SymPy SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python.

2017-04-05

Data collections Updates

  • Expansion of MERRA-2 collection at NCI with the inst3d_3d_asm_Np data. View MERRA-2 page

UM10.6 GA7

LROSE

module use /g/data3/hh5/public/modules module load lrose

Conda Updates

  • ===Several packages have been added to the public Anaconda environments https://accessdev.nci.org.au/trac/wiki/User%20Guides/conda===
    • | metpy: MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.
    • | windspharm: windspharm is a Python package for performing computations on global wind fields in spherical geometry.
    • | python-cdo: Python bindings for CDO
    • | pyart: The Python ARM Radar Toolkit
    • | bokeh: Bokeh is a Python interactive visualization library that targets modern web browsers for presentation.

2017-03-17

Python libraries at NCI

  • We've moved the Python libraries installed with Miniconda to a public space (/g/data3/hh5/public/modules) so there is no need anymore to be part of ACCESS to use it. See the | updated instructions

UM Newsletter

Massdata utility

  • Are you moving data to/from massdata? Are you having issues with the copy stopping in the middle of things and impossibility to restart from where it failed? Aidan wrote a small utility to help with that. It’s available to anyone, info: [[2]]

2017-02-15

MPAS-Atmosphere

  • The latest release of MPAS-Atmosphere has been installed on Raijin. This model will probably supplant WRF sometime in the future.

Python libraries at NCI

  • We've installed some Python libraries using Miniconda in /g/data1/access. The aim is to facilitate access to and management of commonly used Python libraries on Raijin, Accessdev and VDI. It also enables users to locally install their own libraries using Miniconda without duplicating the centrally installed libraries. For the moment, users have to be part of the ACCESS group on Raijin to use these (if you are not, just ask access on my.nci.org.au) but we plan to make it public access. More details are | here.

Sharing facility at NCI

  • We worked at NCI to enable users to share data stored on Raijn with collaborators who don't have access to NCI. This is aimed for work in progress which can not be published and opened publicly. The files are accessible for a short time only to allow your collaborators to download them to their own storage. We are working on making it user-friendly but if you need to share some data in the meantime, please ask climate_help@nci.org.au.

Updating OpenMPI library for MOM

  • After the new quarter maintenance, some MOM jobs experienced some stalling behaviour at the end of the job. This mainly affect large configurations (0.1°). The fix is to recompile MOM using the openmpi/10.2.1 version. If you don't know whether you should worry about it or how to recompile, please contact climate_help@nci.org.au.

NU-WRF

  • The latest manual is now on the CMS wiki. Just follow the NU-WRF link in the navigation sidebar.

2016-08-22

Configurations

UM

  • The Accesscollab UMUI server is no longer supported, and we can't guarantee the safety of data on it. Please move any data on the 'Collab' page of the UMUI to the 'Accessdev' or 'Collab(local)' page. The 'Collab (local)' page contains copies of the Accescollab database from last year, if your jobs are present there no further action is required.

> To move an experiment folder right click on it and select 'copy' then go to the 'accessedev' or 'collab(local)' page and use the 'paste' button in the top right of umuix to copy all jobs in that folder.