Difference between revisions of "Conda"

m (Changed advice to use profile rather than basic)
Line 1: Line 1:
 +
 
== Conda Python Environments ==
 
== Conda Python Environments ==
  
Line 7: Line 8:
 
To use any of the conda environments you must first run
 
To use any of the conda environments you must first run
  
<code>
+
<code>module use /g/data3/hh5/public/modules</code>
module use /g/data3/hh5/public/modules
 
</code>
 
  
You can safely put this in your <code>~/.bashrc</code> file
+
You can safely put this in your <code>~/.profile</code> or <code>~/.login</code> file
  
 
=== Stable Environment ===
 
=== Stable Environment ===
Line 17: Line 16:
 
We update the stable environment once a quarter, around when NCI do their quarterly maintenance of Raijin. Otherwise everything in the environment stays fixed, we don't update packages or install new packages unless something is very broken.
 
We update the stable environment once a quarter, around when NCI do their quarterly maintenance of Raijin. Otherwise everything in the environment stays fixed, we don't update packages or install new packages unless something is very broken.
  
<code>
+
<code>module load conda/analysis3</code>
module load conda/analysis3
 
</code>
 
  
 
=== Unstable Environment ===
 
=== Unstable Environment ===
Line 25: Line 22:
 
The unstable environment gets updated more often, as we install new packages or apply updates to existing ones. If you ask for a new package it will be installed here.
 
The unstable environment gets updated more often, as we install new packages or apply updates to existing ones. If you ask for a new package it will be installed here.
  
<code>
+
<code>module load conda/analysis3-unstable</code>
module load conda/analysis3-unstable
 
</code>
 
  
 
When we do our quarterly update the unstable environment becomes the new stable environment.
 
When we do our quarterly update the unstable environment becomes the new stable environment.
Line 33: Line 28:
 
== Interactive Analysis / Jupyter ==
 
== Interactive Analysis / Jupyter ==
  
For interactive analysis we encourage making use of NCI's [[VDI]] system, which allows you to run Jupyter notebooks without waiting in the queue
+
For interactive analysis we encourage making use of NCI's [[VDI|VDI]] system, which allows you to run Jupyter notebooks without waiting in the queue
  
 
== Requesting new packages ==
 
== Requesting new packages ==
Line 45: Line 40:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
| python || 3.6.7  
+
| python
 +
| 3.6.7
 
|-
 
|-
| numpy || 1.16.4  
+
| numpy
 +
| 1.16.4
 
|}
 
|}
  
 
'''Notable Changes'''
 
'''Notable Changes'''
* arccssive has been renamed to clef
+
 
* A bug preventing Iris from opening UM files without a date has been fixed
+
*arccssive has been renamed to clef  
 +
*A bug preventing Iris from opening UM files without a date has been fixed  
  
 
'''Notable New Packages'''
 
'''Notable New Packages'''
* [https://github.com/kwgoodman/bottleneck bottleneck] Fast rolling operations
+
 
* [https://github.com/ecmwf/cfgrib cfgrib] CF metadata for GRIB files
+
*[https://github.com/kwgoodman/bottleneck bottleneck] Fast rolling operations  
* [https://github.com/SciTools/cf-units cfunits] Convert between CF units
+
*[https://github.com/ecmwf/cfgrib cfgrib] CF metadata for GRIB files  
* [https://github.com/shoyer/h5netcdf h5netcdf] Pythonic interface to netCDF4 via h5py
+
*[https://github.com/SciTools/cf-units cfunits] Convert between CF units  
* [https://github.com/intake/intake intake] Lightweight data catalogues
+
*[https://github.com/shoyer/h5netcdf h5netcdf] Pythonic interface to netCDF4 via h5py  
* [https://gitlab.com/remikz/nccmp/ nccmp] Compare netcdf files
+
*[https://github.com/intake/intake intake] Lightweight data catalogues  
* [https://github.com/pydata/sparse sparse] Sparse multi-dimensional arrays
+
*[https://gitlab.com/remikz/nccmp/ nccmp] Compare netcdf files  
 +
*[https://github.com/pydata/sparse sparse] Sparse multi-dimensional arrays

Revision as of 03:58, 31 July 2019

Conda Python Environments

CMS maintain an Anaconda Python environment at NCI, with a wide variety of climate and weather related libraries.

You can find the most recent list of libraries at our github repository, or run conda list with an environment loaded.

To use any of the conda environments you must first run

module use /g/data3/hh5/public/modules

You can safely put this in your ~/.profile or ~/.login file

Stable Environment

We update the stable environment once a quarter, around when NCI do their quarterly maintenance of Raijin. Otherwise everything in the environment stays fixed, we don't update packages or install new packages unless something is very broken.

module load conda/analysis3

Unstable Environment

The unstable environment gets updated more often, as we install new packages or apply updates to existing ones. If you ask for a new package it will be installed here.

module load conda/analysis3-unstable

When we do our quarterly update the unstable environment becomes the new stable environment.

Interactive Analysis / Jupyter

For interactive analysis we encourage making use of NCI's VDI system, which allows you to run Jupyter notebooks without waiting in the queue

Requesting new packages

You can ask for a new package to be installed or for an existing package to be updated by emailing cws_help@nci.org.au.

Update History

19.04 (Stable from 2019-07-02)

python 3.6.7
numpy 1.16.4

Notable Changes

  • arccssive has been renamed to clef
  • A bug preventing Iris from opening UM files without a date has been fixed

Notable New Packages

  • bottleneck Fast rolling operations
  • cfgrib CF metadata for GRIB files
  • cfunits Convert between CF units
  • h5netcdf Pythonic interface to netCDF4 via h5py
  • intake Lightweight data catalogues
  • nccmp Compare netcdf files
  • sparse Sparse multi-dimensional arrays