Running Jupyter Notebook

On VDI

Currently, the easiest way to run IPython Notebook is on NCI's Virtual Desktop Infrastructure (VDI). For a guide to set up and use VDI, click here.

Within VDI, open a Linux terminal (Applications menu -> System Tools -> Terminal). Inside the Terminal, load the conda environment

module use /g/data/hh5/public/modules
module load conda

You should then be able to start the notebook with

jupyter notebook

Connect your local browser

You can connect the browser on your computer to the jupyter notebook running on vdi once it's started.

You still need to connect to VDI in the usual way. Once the session is started, open a terminal. You need the name of the computer that is running your VDI instance. It is of the form vdi-nXX where XX is a number. It is possible that this is already displayed in your command line prompt there, otherwise you can run the command:

hostname

and it will give you the hostname. Remember it.

Next, you load the conda module as above, but tell jupyter to not start a browser:

module use /g/data/hh5/public/modules
module load conda
jupyter notebook --no-browser

You wait for this line:

http://localhost:8888/?token=...

The 4 digits after the colon (often just 8888, but not always) are the port. We use this number and the hostname to create a new ssh connection to VDI. So in a new terminal session on your computer, run the command:

ssh userXX@vdi-nXX.nci.org.au -L 8888:localhost:8888

Of course, you need to replace userXX with your NCI username, vdi-nXX with the hostname, and both instances of 8888 with the actual port number.

Once you've done that, you can copy the full line that you got from jupyter notebook (the one http://localhost:8888/?token=... (including the full token) into your browser of choice.

On Gadi

You can also run a notebook from Gadi's compute nodes using the gadi_jupyter script.

Notes

• You need to download this script to your own computer and execute there. The script handles the connection.
• This job requires an implementation of bash. This is default on MacOS and Linux, but not Windows.
• This will cost SU depending on the queue and the resources requested (default: 1 cores, express queue)
• You also won't have access to the internet to, for example, download data.

Running

You can download the script from the link above directly, or by running

git clone https://github.com/coecms/nci_scripts.git

which will download the script into a directory called nci_scripts.

There are several options that you can use when running the script:

General Options:
    -h:         Print help
    -l:         NCI username
    -L:         NCI login node (default 'gadi.nci.org.au')
    -e:         Conda environment

Queue Options:
    -q QUEUE:   Queue name
    -n NCPU:    Use NCPU cpus
    -m MEM:     Memory allocation (default 4*NCPU GB)
    -t TIME:    Walltime limit (default 1 hour)
    -J JOBFS:   Jobfs allocation (default 100 GB)
    -P PROJ:    Submit job under project PROJ