Revision as of 21:37, 21 January 2021 by C.carouge (talk | contribs) (FAQ miscellaneous)

This page lists FAQs for the different model supported by the CMS team, as well as FAQs around accessing and publishing data.

FAQ pages

FAQ miscellaneous

How to debug issues with gadi_jupyter?

The script submitted to PBS and the PBS logs are saved under the directory tmp/runjp directory under the user's /scratch directory for the group used to run the script. These will often have more information if the script was started alright but you can't seem to be able to connect to the Jupyer Notebooks. This usually happens when the job started alright but exited due to an error in the script.

For example, let's say I (ccc561) requests gadi_jupyter to run using the project w35. The information will then be stored under /scratch/w35/ccc561/tmp/runjp

If I then run gadi_jupyter again but this time running on w40, the information will be stored under /scratch/w40/ccc561/tmp/runjp

How do I run independent jobs in parallel?

If you have to run the same script with a lot of independent sets of inputs, or you have lots of similar scripts to run at NCI. And these scripts run on 1 processor. You could submit each of your jobs independently via PBS but then small jobs are not efficient at NCI. It would probably be better for you to have a master script that dispatches your jobs to available processors. You then submit this master script to PBS and this script requires a lot more processors and hence might be more efficient with the scheduler. There is an example of such a master script on this page.

How to transform 1D lat/lon arrays into 2D lat/lon arrays with Python?

If using numpy or masked arrays, one can use the resize() method. Assuming slat and slon are 1D latitude and longitude arrays, do this:

tmp    = numpy.resize(slat, (slon.size, slat.size))
slat2D = numpy.transpose(tmp)
slon2D = numpy.resize(slon, (slat.size, slon.size))

If you want masked arrays, replace "numpy." by "numpy.ma."

How do I push my local code changes on raijin back to github using ssh keys?

You have an account on | github. You have cloned a repository to raijin and made changes to it. Now you want to push those changes back to the github repository but you get errors like this (where <username> is your github username):

error: The requested URL returned error: 403 Forbidden while accessing https://github.com/<username>/test.git/info/refs

fatal: HTTP request failed

Firstly, you need to | make an ssh key and add the public part of the key to your account on github.

Check the url to which you are trying to push your changes:

git remote -v
origin https://github.com/<username>/test.git (fetch)
origin https://github.com/<username>/test.git (push)

If it looks like the output above you need to alter the url and put in a login name in front of the address. For ssh access the login name is always git. The command to set the remote url is:

git remote set-url origin ssh://git@github.com/<username>/test.git

but replace <username> with your github username.

Why do I get segfaults?

A segmentation fault means the program you are using has tried to read or write a memory location it does not have access to. This can mean there is an error in the code, or there is a system limit set. If the program runs for others, but not you, it may well be a system limit issue. This seems to be quite common when trying to write netcdf files. In some cases this can be overcome by setting your stack size to unlimited

For tcsh/csh add this to your .cshrc:

limit stacksize unlimited

For bash add this to your .bashrc:

ulimit -s unlimited

Note: you must log out and log back in again for this change to take effect.