Running Jupyter Notebook
When logging into the OOD, you will have an option to open Jupyter Lab. Please follow the instructions to request the best sized cluster for your work.
You can also run a notebook from Gadi's compute nodes using the gadi_jupyter script.
For this method to work, you must become part of the hh5 project at NCI using my.nci.org.au.
This project will give you access to conda environments for Python managed by the CMS team.
Setting Up SSH
You will need to set up SSH keys to use gadi_jupyter
Create the file ~/.ssh/config with something like the following lines in:
Host *.nci.org.au User abc123 # Your NCI username here
Create a SSH key by running 'ssh-keygen'. It will ask for a 'passphrase', make sure you enter one, it doesn't need to be the same as your NCI password. Use the default key file name.
Copy your SSH key to gadi. Some computers have a command that can do this automatically - 'ssh-copy-id gadi.nci.org.au'. If not, you need to add the contents of the file '~/.ssh/id_rsa.pub' on your local computer to the end of the file '~/.ssh/authorized_keys' on Gadi.
Enable your SSH key by running the command 'ssh-add', it will ask for the SSH passphrase you entered when you made the key (which may not be your NCI password). If it says 'Could not open a connection to your SSH agent', or '‘could not connect to authentication agent', run 'ssh-agent bash' and then 'ssh-add'.
You can now run './gadi_jupyter'
- You need to download this script to your own computer and execute there. The script handles the connection.
- This job requires an implementation of bash. This is default on MacOS and Linux, but not Windows.
- This will cost SU depending on the queue and the resources requested (default: 1 cores, express queue)
- You also won't have access to the internet to, for example, download data.
You can download the script from the link above directly, or by running
git clone https://github.com/coecms/nci_scripts.git
which will download the script into a directory called nci_scripts.
There are several options that you can use when running the script:
General Options: -h: Print help -l: NCI username -L: NCI login node (default 'gadi.nci.org.au') -e: Conda environment Queue Options: -q QUEUE: Queue name -n NCPU: Use NCPU cpus -m MEM: Memory allocation (default 4*NCPU GB) -t TIME: Walltime limit (default 1 hour) -J JOBFS: Jobfs allocation (default 100 GB) -P PROJ: Submit job under project PROJ
To run gadi_jupyter on Windows you'll need Bash and some form of SSH available. This could be from
* git bash - https://gitforwindows.org/ * cygwin terminal - https://www.cygwin.com/
Install one of these, then within the terminal follow the 'Setting up SSH' instructions before running 'gadi_jupyter'. (The terminal should start in your 'My Documents' directory if you need to find the downloaded script)