Difference between revisions of "ERA INTERIM"

 
Line 17: Line 17:
  
 
We found very high values for temperature (~330-350 K) near the lower troposphere in the following periods: 26-31 Dec 2002 and 21-25 Dec 2003. Other variables might be similarly affected.
 
We found very high values for temperature (~330-350 K) near the lower troposphere in the following periods: 26-31 Dec 2002 and 21-25 Dec 2003. Other variables might be similarly affected.
 +
  
 
=== '''License''' ===
 
=== '''License''' ===
  
<span style="background-color: #ffffff;">To comply with the ECMWF licensing you need to be registered with the ECMWF to use this data.</span> <span style="background-color: #ffffff;">To register with the ECMWF you have to </span><span style="background-color: #ffffff; line-height: 1.5;">sign the </span>[http://apps.ecmwf.int/datasets/data/interim_full_daily/licence/ ERA Interim license and agreement]<span style="background-color: #ffffff; line-height: 1.5;">.</span><br/> <br/> <span style="background-color: #ffffff;">'''NB''': Users of the ECMWF data sets are requested to reference the source of the data in any publication, e.g. "''ECMWF ERA-Interim data used in this study/project have been provided by ECMWF/have been obtained from the ECMWF MARS Data Server on <date>''".</span><br/> You also have to add the following paper to your references:<br/> [https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.828 https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.828]
+
<span style="background-color: #ffffff;">To comply with the ECMWF licensing you need to be registered with the ECMWF to use this data.</span> <span style="background-color: #ffffff;">To register with the ECMWF you have to </span><span style="background-color: #ffffff; line-height: 1.5;">sign the </span>[http://apps.ecmwf.int/datasets/data/interim_full_daily/licence/ ERA Interim license and agreement]<span style="background-color: #ffffff; line-height: 1.5;">.</span><br/> &nbsp;
 +
 
 +
To acknowledge the ERA-Interim data, please refer to the ERA-Interim&nbsp;[https://apps.ecmwf.int/datasets/data/interim-full-daily/licence/ licence]&nbsp;for details on the wording to use.
 +
 
 +
To cite the source of the data, you may use the following data citation (as part of the bibliography):
 +
 
 +
European Centre for Medium-range Weather Forecast (ECMWF) (2011): The ERA-Interim reanalysis dataset, Copernicus Climate Change Service (C3S) (accessed&nbsp;''<insert date of access here''>), available from&nbsp;[https://www.ecmwf.int/en/forecasts/datasets/archive-datasets/reanalysis-datasets/era-interim https://www.ecmwf.int/en/forecasts/datasets/archive-datasets/reanalysis-datasets/era-interim]&nbsp;
 +
 
 +
If no specific advice is given by the journals, it is usually recommended that the above data citation is put in the acknowledgements section.
 +
 
 +
<br/> You also have to add the following paper to your references:<br/> [https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.828 https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.828]
  
 
=== '''Data access''' ===
 
=== '''Data access''' ===

Latest revision as of 23:30, 25 August 2022

A subset of the ERA Interim re-analysis data ranging from 1979 to the 31st of August, when ERA Interim production ceased is replicated on NCI server. We downloaded the original grib data from the ECMWF server , specific variables were then extracted and converted to monthly netcdf files. For more information on the original ECMWF collection visit their website: ERA-Interim page

Known issues

There are several known quality issues for the ERA Interim data, here is a list of web resources:

ERA Interim quality issues from ECMWF website

Climate Data Guide page on ERAI

Overview and re-analysis comparison from reanalyses.org

Some of these analysis are described more in depth in the following article: The ERA-Interim reanalysis: configuration and performance of the data assimilation system

We are listing all the issues we are aware of in this ERAI known issues page, please feel free to contribute, we just provide the data as it is downloaded from the ECMWF server, we do checks to make sure that our data is the same as the original but we cannot run checks for the original data.

We found very high values for temperature (~330-350 K) near the lower troposphere in the following periods: 26-31 Dec 2002 and 21-25 Dec 2003. Other variables might be similarly affected.


License

To comply with the ECMWF licensing you need to be registered with the ECMWF to use this data. To register with the ECMWF you have to sign the ERA Interim license and agreement.
 

To acknowledge the ERA-Interim data, please refer to the ERA-Interim licence for details on the wording to use.

To cite the source of the data, you may use the following data citation (as part of the bibliography):

European Centre for Medium-range Weather Forecast (ECMWF) (2011): The ERA-Interim reanalysis dataset, Copernicus Climate Change Service (C3S) (accessed <insert date of access here>), available from https://www.ecmwf.int/en/forecasts/datasets/archive-datasets/reanalysis-datasets/era-interim 

If no specific advice is given by the journals, it is usually recommended that the above data citation is put in the acknowledgements section.


You also have to add the following paper to your references:
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.828

Data access

You can then request to join the ub4 project by using your NCI account to access My NCI portal.
Remember you have to register with ECMWF before. Be aware that by requesting to be part of the ub4 project you are automatically agreeing to the ECMWF license agreement terms.

 

The data is available on gadi under /g/data/ub4/erai/grib for the grib version and /g/data/ub4/erai/netcdf for the netcdf version.

NB  that we download all the available surface fields, both analysis and forecast, but we don't convert all of them to netcdf, unless they are specifically requested.

The netcdf version is organized in monthly field files, while the grib files have all the fields in one file.

The current netcdf version was released in April 2015, the previous version hosted on ua8 is not anymore available. If you requested access before we moved the data to ub4, you are already part of ub4 and don't need to request access again to use the data.

Monthly averaged fields are available in ua8 as part of the CREATE-IP temporary copy hosted in

/g/data/ua8/synda/CREATE-IP/reanalysis/ECMWF/ERA-Interim/

This is a temporary replica , eventually NCI should provide the CREATE-IP dataset in a different project, see this page for details.

If you are having trouble getting access or the data you are looking for is not available there, let us know by e-mailing cws_help<at>nci.org.au .

Current netcdf version: v1.0

The main differences with the previous version, which is not anymore available, are:

1) netcdf4 format with compression, which allowed us to get rid of scale and offset

2) new filenames of the form <varname>_<frequency>_ERAI_historical_<level>_<from-date>_<to-date>.nc

Ex. : ta_6hrs_ERAI_historical_an-ml_20110101_20110131.nc

3) some of the variable names have changed to the correspondent CMIP5 standard name, all names are now lower cases

4) new directory structure /g/data/ub4/erai/netcdf/<frequency>/<realm>/<level>/<version>/<variable>/files…

Ex. : /g/data/ub4/erai/netcdf/6hr/atmos/oper_an_ml/v01/ta/

5) there is more metadata information in the files, in particular we adopted CF conventions and added standard_names wherever possible. We also added an attribute MD5 checksum for each variable, this can be use to check for data corruption by using ncks or nco

Most of the data will be in the "atmos" realm, with the exception of ERAI land which is is "land" and few ocean fields. And most files have "6hr" frequency, with the exception of the forecast data which has "3hr" frequency.

The scripts used to convert the files from grib to netcdf are available in the checks github.
https://github.com/coecms/CollectionsScripts