Getting started on Windows

Revision as of 01:19, 7 April 2021 by S.wales (talk | contribs) (Cygwin)

Windows computers work a little different to Linux and Mac, which both share a common UNIX ancestry. While you can achieve similar results using Power Shell in Windows to Bash in Linux/Mac, the way it works is completely different so you have to learn a new way of doing things when connecting to facilities like NCI. For that reason we recommend setting up a Linux-like environment on your Windows computer, so that everything works in a similar way.

You should be able to:

  • Connect to NCI to access models and data
  • Develop, test and run Python, Fortran etc. programs locally
  • Use a common interface on your local computer and NCI

For more information on how to use the terminal and Bash, check out the Software Carpentry Bash course

SSH configuration

Regardless of the method you use, you will connect to remote servers (Gadi, VDI, Accessdev) using SSH. It is worth setting up SSH keys and a SSH configuration file. You can find detailed instructions on the induction page or in our SSH training video.

NCI Virtual Desktop (VDI)

NCI's virtual desktops (VDI) are an easy way to connect to NCI facilities from desktop computers. VDI provides a Linux desktop in a window, running on NCI's cloud

Check NCI's VDI documentation for the current installation and setup instructions https://vdi.nci.org.au/help, and the Getting started on Linux page for details on using the Linux environment

Advantages:

  • Access to NCI's /g/data filesystem
  • Conda and Matlab (see NCI documentation) available
  • No compute time/SU charge when running analyses on VDI
  • Doesn't require a powerful home computer

Disadvantages:

  • VDI servers are shared between multiple users, can run out of memory
  • Requires network connection - can't use VDI on a plane etc.

Windows Subsystem for Linux (WSL)

Windows subsystem for Linux (WSL) is a way to have a Linux environment on your Windows computer, without having to install and switch between different operating systems.

For up-to-date WSL installation instructions, see Microsoft's documentation at https://docs.microsoft.com/en-us/windows/wsl/install-win10. There are a few different Linux variants you can install, a good starting point is Ubuntu if you can't decide.

Once you have WSL and a Linux environment set up you can use Linux commands to install further programs, e.g. for Ubuntu

sudo apt install vim

will install the vim text editor.

With WSL installed you can then follow the Getting started on Linux instructions to set up SSH etc.

Advantages:

  • Runs on your own computer
  • Install whatever programs you need

Disadvantages:

  • Large size (~2GB for base install, another GB or so for Anaconda)
  • Remote X11 windows can be slow if you have a bad connection

Windows Terminal

Windows Terminal is a nicer command line interface for Windows than the default Power Shell program, with more customisation options. It integrates with WSL, letting you use it to start Bash terminal sessions

Windows terminal can be installed through the Windows app store, further details at https://docs.microsoft.com/en-us/windows/terminal/

X11

In order to show remote windows from Gadi on your own screen you'll need an X11 server program running on your computer.

If X11 isn't running, you will get errors when starting programs that bring up a new window, e.g:

saw562@gadi-login-04 ~ $ xclock
Error: Can't open display:

There are a few different X11 server programs available for Windows, including:

* https://sourceforge.net/projects/vcxsrv/

VSCode & Power Shell SSH

VS Code is a nice code editor that can run on Windows, Mac and Linux. You can edit code locally, or use plugins to remotely connect to servers over SSH. It also integrates with WSL allowing you to edit code in the Linux environment, and with Anaconda, letting you run Python programs in different environments.

Windows includes its own version of SSH that you can use directly from Power Shell. Visual Studio Code can use this to connect to Gadi from your Windows computer. You will need some special options when creating a SSH key with Power Shell SSH:

ssh-keygen -t ecdsa -b 521 -f .ssh/id_ecdsa

Power shell doesn't have the ssh-copy-id script, so you'll need to add the contents of the new public key file .ssh/id_ecdsa.pub to the ~/.ssh/authorized_keys file on Gadi manually.

The Power Shell SSH config file is stored in your user directory, e.g. C:\Users\Scott\.ssh\config. Settings work just like Linux, e.g.:

Host gadi
  HostName gadi.nci.org.au
  User saw562 # Replace with your own username
  ForwardAgent yes

You can also use the Power Shell SSH to connect to Gadi, bypassing WSL, but that may have problems running X11 windows

https://code.visualstudio.com/

Software

NCI Jupyter

Jupyter provides a 'notebook' interface for working with Python - you can combine Python code, text, latex equations and plots in a web interface.

The Centre has developed scripts that can run Jupyter on NCI facilities - VDI or Gadi - and display the notebook interface on your local computer. These scripts are available at https://github.com/coecms/nci_scripts, see the instructions there for usage.

You can also run Jupyter directly on VDI, by loading the Conda environment and running 'jupyter lab'.

On Windows, the jupyter scripts must be run through a Bash terminal (From WSL or Cygwin).

Anaconda Python / Rstudio

We recommend using Anaconda if you're working with Python on your own computer. This makes it easy to install packages like numpy and netCDF4, which depend on binary libraries. Anaconda uses the same system we use for the Conda environments at NCI. Anaconda also supports Rstudio if you work with R.

You can install the Windows native version, or the Linux version in WSL. Normally more packages are available for Linux, but commonly used ones will be available for either. We recommend enabling the 'conda-forge' package repository, you will find a wider variety of packages there and they are generally well supported.

Matlab

Check with your local IT for information on how to use Matlab, they will have information on how to connect to their Matlab license server.

You can also use Matlab on NCI's systems via VDI, however you need to use your university license.

Other Alternatives

Putty

Putty is an alternative SSH implementation for Windows

https://www.putty.org/

Cygwin

Cygwin works similarly to WSL, it lets you run Linux programs on a Windows computer

https://www.cygwin.com/

Git for windows includes a Bash terminal based on Cygwin as well

https://gitforwindows.org/

File transfers

It is sometimes useful to transfer files between NCI servers and your local computer. You can do so via the command line with programs like rsync which is readily available. If you prefer a graphical interface, NCI advises you can install a program like Cyberduck. You can then use a SFTP connection. To access files on Gadi, it is preferable to use the data mover nodes: gadi-dm.nci.org.au for the server's name.