Difference between revisions of "Running Jupyter Notebook"

Line 1: Line 1:
 
=On VDI=  
 
=On VDI=  
  
Currently, the easiest way to run IPython Notebook is on NCI's [http://nci.org.au/services/vdi/ | Virtual Desktop Infrastructure (VDI)]. For a guide to set up and use VDI, click [https://opus.nci.org.au/display/Help/VDI+User+Guide | here].
+
Currently, the easiest way to run IPython Notebook is on NCI's [[VDI|Virtual Desktop Infrastructure (VDI)]]. For a guide to set up and use VDI, click [https://opus.nci.org.au/display/Help/VDI+User+Guide | here].
  
 
Within VDI, open a Linux terminal ('''Applications menu''' -> '''System Tools''' -> '''Terminal''').
 
Within VDI, open a Linux terminal ('''Applications menu''' -> '''System Tools''' -> '''Terminal''').
Line 14: Line 14:
 
jupyter notebook
 
jupyter notebook
 
</syntaxhighlight>
 
</syntaxhighlight>
<span style="background-color: #ffffff; color: #333333; display: block; font-family: &quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif; font-size: 14px;">
 
----
 
</span><span style="background-color: #ffffff; display: block; font-family: Arial,Helvetica,sans-serif;">
 
=On Raijin=
 
  
If you don't have access to VDI, you can still run IPython Notebook from Raijin in a browser on your local computer. Currently these instructions show how to run the notebook from the login node - '''which is not recommended''' - and, as such, are just a demonstration of what is possible. Ideally the notebook should be run from an interactive session.
+
=On Gadi=  
</span>
 
Log in to raijin using SSH, forwarding your ports as you go (you might have to sudo this)
 
<syntaxhighlight lang=bash>
 
ssh -L 8889:localhost:8889 [USERNAME]@raijin.nci.org.au
 
</syntaxhighlight>
 
Load the conda environment
 
<syntaxhighlight lang=bash>
 
module use /g/data3/hh5/public/modules
 
module load conda
 
</syntaxhighlight>
 
 
 
Set up an ipython profile that serves your notebooks (adapted from [[https://ipython.org/ipython-doc/dev/notebook/public_server.html]])
 
<syntaxhighlight lang=bash>
 
ipython profile create nbserver
 
</syntaxhighlight>
 
 
 
Then modify the Notebook config file in ~/.ipython/profile_nbserver/ipython_config.py (or create that file if it wasn't generated) so that it reads:
 
<syntaxhighlight lang=python>
 
c = get_config()
 
c.NotebookApp.open_browser = False
 
# It is a good idea to put it on a known, fixed port - must match port used to ssh in.
 
c.NotebookApp.port = 8889
 
c.NotebookApp.base_project_url = 'ipython'
 
 
 
# The following can be used to set up a password,
 
# from here: https://ipython.org/ipython-doc/dev/notebook/public_server.html
 
# c.NotebookApp.certfile = u'/absolute/path/to/your/certificate/mycert.pem'
 
# c.NotebookApp.ip = '*'
 
# c.NotebookApp.password = u'sha1:bcd259ccf...[your hashed password here]'
 
 
 
</syntaxhighlight>
 
The dependencies required for ipython have been installed and should load automatically when the ipython module is loaded.
 
 
 
You should be able to start the notebook with
 
<syntaxhighlight lang=bash>
 
ipython notebook --profile=nbserver
 
</syntaxhighlight>
 
  
On your local computer, direct your browser to [[http://127.0.0.1:8889/ipython/]] and you should have access to the notebook.
+
You can also run a notebook from Gadi's compute nodes using the [https://github.com/COSIMA/cosima-cookbook/blob/master/scripts/jupyter_gadi.sh jupyter_gadi.sh] script. Note that unlike VDI this does cost SU allocation, as it is running on the supercomputer nodes. You also won't have access to the internet to download data.
 
 
If another user is already running an ipython session using the port number above, the ipython process will tell you with a message like this:
 
<syntaxhighlight>
 
[I 11:06:03.545 NotebookApp] The port 8899 is already in use, trying another random port.
 
</syntaxhighlight>
 
and it will tell you which port it has chosen instead:
 
<syntaxhighlight>
 
[I 11:06:03.552 NotebookApp] The IPython Notebook is running at: http://localhost:8900/ipython/
 
</syntaxhighlight>
 
 
 
Unfortunately the ssh connection will only forward the port you specified when you logged in. You should edit the ~/.ipython/profile_nbserver/ipython_config.py file, change the port number to the one ipython chose (or another higher number less than 65535), log out and log back in, using the new port number.
 
<syntaxhighlight lang=bash>
 
ssh -L 8889:localhost:8890 [USERNAME]@raijin.nci.org.au
 
</syntaxhighlight>
 
The first port number is the one you connect to on your local computer, and can remain unchanged.
 
  
 
[[Category:Python]]
 
[[Category:Python]]

Revision as of 23:50, 4 December 2019

On VDI

Currently, the easiest way to run IPython Notebook is on NCI's Virtual Desktop Infrastructure (VDI). For a guide to set up and use VDI, click | here.

Within VDI, open a Linux terminal (Applications menu -> System Tools -> Terminal). Inside the Terminal, load the conda environment

module use /g/data3/hh5/public/modules
module load conda

You should then be able to start the notebook with

jupyter notebook

On Gadi

You can also run a notebook from Gadi's compute nodes using the jupyter_gadi.sh script. Note that unlike VDI this does cost SU allocation, as it is running on the supercomputer nodes. You also won't have access to the internet to download data.