Difference between revisions of "ERA5"

m (Updated information on available data)
 
(36 intermediate revisions by 3 users not shown)
Line 1: Line 1:
ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the [http://climate.copernicus.eu/ | Copernicus Climate Change Service (C3S)].
 
Data processing for ERA5 is carried out by [http://www.ecmwf.int/ | ECMWF], using ECMWFS' [https://www.ecmwf.int/en/research/modelling-and-prediction | Earth System model IFS], cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.
 
  
We are providing a small subset of ERA5 while waiting for a longer term solution to store and manage this big dataset.
+
'''<span style="color:#c0392b;">NCI is now mananging the official replica for this dataset. The data in ub4 is not anymore available, we removed access and we will delete the actual files in June.</span>'''
Currently data is available from 1979 to present. We have covered the period 2000-2018 for all the requested variables and we are in the process of downloading the other years, refer to the ERA% variable list for more details.
 
ERA5 comprises different data stream, we have only a small subset of the surface operative analysis. For the moment we are not considering downloading model or pressure level data as the size of these are quite big and we wouldn't be able to provide enough data to use them to run a model.
 
For more information on ERA5 data refer to the [https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation | ECMWF website].
 
  
If you need other variables for your current research then you can contact us at climate_help@nci.org.au
+
'''<span style="color:#c0392b;">For any information on the new collection and transition:&nbsp;</span><span style="color:#c0392b;">[https://opus.nci.org.au/display/ERA5/ERA5+Community+Home ERA5 Community site].</span><span style="color:#c0392b;">&nbsp;For any direct enquiry: help@nci.org.au .</span>'''
'''NB''' : this is a temporary fix so we will download only variables which are useful for a variety of projects and that are needed now, if you can use ERA-Interim instead or you will be using ERA5 in the future but not in the next few months, then we won't consider your request.
 
  
===Requests constraints===
+
'''<span style="color:#c0392b;">NB. any enquiry via the CWS helpdesk will have to be transferred to the right queue&nbsp;and would be delayed!</span>'''
  
<span class="s1">We will only accept requests for:</span>
+
ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the [http://climate.copernicus.eu/ Copernicus Climate Change Service (C3S)].
<span class="s1"> - data on regular 0.25 by 0.25 lat/lon grid</span>
 
<span class="s1"> - global region for surface variables </span>
 
<span class="s1">- Austral-Asian region for pressure level. The region includes Australia and the Maritime continent and the CORDEX domain and is defined as:</span>
 
<span class="s1">Latitude: 20 to -57</span>
 
<span class="s1">Longitude: 78 to 220</span>
 
<span class="s1"> NB longitude for the global region is -180 to 180</span>
 
<span class="s1"> - analysis data and no ensemble data (analysis includes surface forecast fields such as precipitation). It is currently impossible to provide input data for a climate model</span>
 
<span class="s1">You can submit a request for downloading following these criteria. We will only consider requests that specify the exact code/stream etc. You can refer for more information to the documentation here and to the <span class="s3">[https://protect-au.mimecast.com/s/-dp6CZYM5DFXmX6BIzaRdX?domain=apps.ecmwf.int | ERA5 catalogue]</span>. Be as specific as possible in your request: grib or netcdf format? which forecast steps do you need if you request forecast data? (see ERA5 website) </span>
 
  
===Known Issues===  
+
You can use the [https://cds.climate.copernicus.eu/cdsapp#!/software/app-era5-explorer?tab=app ERA5 Explorer] tool to get a preview of the data (only few variables currently available). Or you can use the [https://cds.climate.copernicus.eu/toolbox/doc/index.html# CDS toolbox] to do some of your analysis without download the data. As descirbed on the CDS website the toolbox is&nbsp;"Free and available to everyone, the Toolbox links raw data to online computing power through a programming interface. In your own online workspace you can create applications in Python (a programming language) and run them on the CDS computers, allowing you to retrieve the data you are interested in, make the calculations you require and display the results in the format that suits your needs. You can download graphs, maps and data, and also share your online creations with other users."
[https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation#ERA5datadocumentation-Knownissues | Known issues from ECMWF website]
+
 
 +
Data processing for ERA5 is carried out by [http://www.ecmwf.int/ ECMWF], using ECMWFS' [https://www.ecmwf.int/en/research/modelling-and-prediction Earth System model IFS], cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.
 +
 
 +
For more information on ERA5 data refer to the [https://confluence.ecmwf.int/display/CKB/ERA5+data+documentation ECMWF ERA5 documentation].
 +
 
 +
<span class="s1">For more specific information on ERA5 Land: [https://confluence.ecmwf.int/display/CKB/ERA5-Land:+data+documentation ECMWF ERA5 Land documentation].</span>
 +
 
 +
=== '''ERA5 derived products''' ===
 +
 
 +
Recently ECMWF has released new ERA5&nbsp;derived products.&nbsp;These ranged from new variables&nbsp;calculated from ERA5 as for the fire indices or bias -corrected, interpolated versions of ERA5 main variables themselves.&nbsp;For a list check the [https://cds.climate.copernicus.eu/cdsapp#!/search?text=ERA5 Copernicus CDS].
 +
 
 +
==== '''CEMS-Fire - Fire danger indices historical data from the Copernicus Emergency Management Service''' ====
 +
 
 +
This ERA5 derived product was recently added to the Climate Data Store. It&nbsp;provides complete historical reconstruction of meteorological conditions favourable to the start, spread and sustainability of fires. The fire danger metrics provided are part of a vast dataset produced by the Copernicus Emergency Managment Service for the European Forest Fire Information System (EFFIS). The European Forest Fire Information System incorporates the fire danger indices for three different models developed in Canada, United States and Australia. In this dataset the fire danger indices are calculated using weather forecast from historical simulations provided by ECMWF ERA5 reanalysis&nbsp;(see more on [https://cds.climate.copernicus.eu/cdsapp#!/dataset/cems-fire-historical?tab=overview CDS website]). They have a daily temporal resolution on a&nbsp;0.25X0.25 degrees grid. Currently they're available for the same period as ERA5 1979 to present.
 +
 
 +
==== '''WFDE5 - Near surface meteorological variables from 1979 to 2018 derived from bias-corrected reanalysis''' ====
 +
 
 +
This dataset provides bias-corrected reconstruction of near-surface meteorological variables derived from the fifth generation of the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric reanalyses (ERA5). It is intended to be used as a meteorological forcing dataset for land surface and hydrological models&nbsp;(see&nbsp;more on the [https://cds.climate.copernicus.eu/cdsapp#!/dataset/derived-near-surface-meteorological-variables?tab=overview CDS website]).
 +
 
 +
&nbsp;
 +
 
 +
'''Known Issues'''
 +
 
 +
[https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation#ERA5datadocumentation-Knownissues Known issues from ECMWF website]
 +
 
 +
Issue affecting WFDE5 v1.0 &nbsp;"Surface downwelling shortwave radiation" data.&nbsp;This is resolved in v1.1.
  
 
Dec 7 2018: an error has been discovered affecting the values of the following variables
 
Dec 7 2018: an error has been discovered affecting the values of the following variables
{|  
+
 
| '''CDS Dataset''' || [https://protect-au.mimecast.com/s/hEqUClxwAVHDmEV9F9ILGT?domain=cds.climate.copernicus.eu | ERA5 hourly data on pressure levels from 2000 to present]
+
{|
 
|-
 
|-
| '''Parameters affected''' || U-component of wind, V-component of wind
+
| '''CDS Dataset'''
 +
| ERA5 hourly data on pressure levels from 2000 to present]
 
|-
 
|-
| '''Level''' || All pressure levels
+
| '''Parameters affected'''
 +
| U-component of wind, V-component of wind
 
|-
 
|-
| '''Time periods affected''' || 01 Jan. 2000 - 31 Dec. 2008<br>01 Jun. 2018 - present
+
| '''Level'''
 +
| All pressure levels
 
|-
 
|-
 +
| '''Time periods affected'''
 +
| 01 Jan. 2000 - 31 Dec. 2008<br/> 01 Jun. 2018 - present
 
|}
 
|}
 +
 
This problem was caused by the computation of U/V from Vorticity (Vo) and Divergence (D) already interpolated onto a regular grid in CDS. U/V should have been computed from Vo and D in the spherical harmonics representation (the native model representation) and then interpolated to a regular grid before saving to the climate data store.
 
This problem was caused by the computation of U/V from Vorticity (Vo) and Divergence (D) already interpolated onto a regular grid in CDS. U/V should have been computed from Vo and D in the spherical harmonics representation (the native model representation) and then interpolated to a regular grid before saving to the climate data store.
  
The problem is now fixed and we are in the process of re-downloading the data.
+
The problem is now fixed and we have re-downloading&nbsp;the data, download should be completed by the 19 March 2019. From the CDS communication: "<span class="s1">During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a <span class="s2">[https://protect-au.mimecast.com/s/vf55CQnzVqt4vDL7hxh02-?domain=confluence.ecmwf.int known issue]</span>. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."</span>
From the CDS communication:
 
"<span class="s1">During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a <span class="s2">[https://protect-au.mimecast.com/s/vf55CQnzVqt4vDL7hxh02-?domain=confluence.ecmwf.int | known issue]</span>. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."</span>
 
  
===License===
+
<span class="s1">[[Era5_issues|Issues and other feedback shared by users]]: you can check this page for issues and other information shared by other users of ERA5, feel free to add your own contribution.</span>
<span style="background-color: #ffffff; color: #172b4d; font-family: -apple-system,system-ui,&#39;Segoe UI&#39;,Roboto,Oxygen,Ubuntu,&#39;Fira Sans&#39;,&#39;Droid Sans&#39;,&#39;Helvetica Neue&#39;,sans-serif; font-size: 14px;">ERA5 data is open access and free to download for all uses, including commercial use.</span>
 
As for ERAI and other ECMWF datasets, though, you need to have an [https://apps.ecmwf.int/registration/ | account with ECMWF] and accept their terms and conditions, in this particular case ERA5 is covered by the [http://apps.ecmwf.int/datasets/licences/copernicus/ | Copernicus license]
 
  
===ERA5 on raijin===  
+
=== '''License''' ===
  
===='''Surface'''====
+
<span style="background-color: #ffffff; color: #172b4d; font-family: -apple-system,system-ui,'Segoe UI',Roboto,Oxygen,Ubuntu,'Fira Sans','Droid Sans','Helvetica Neue',sans-serif; font-size: 14px;">ERA5 data is open access and free to download for all uses, including commercial use.</span> As for ERAI and other ECMWF datasets, though, you need to&nbsp;[https://apps.ecmwf.int/registration/ register&nbsp;with ECMWF] and accept their terms and conditions, in this particular case ERA5 is covered by the [http://apps.ecmwf.int/datasets/licences/copernicus/ Copernicus license]
data is downloaded as netcdf on a regular 0.25x0.25 longitude/latitude global grid at 1 hr resolution, as available from the Copernicus server. These are the maximum spatial and temporal resolutions possible for this dataset.
 
  
/g/data/ub4/era5/netcdf/surface/<variable>/<yr>/<files>
+
=== &nbsp; ===
  
<span class="s1">filename:</span>
+
==== '''Note on forecasts''' ====
<variable><span class="s1">_era5_global_<from-date>_<to-date>.nc</span>
 
  
===='''Pressure levels'''====
+
<span class="s1">Forecasts are accumulated values defined at 20 steps (0-19) starting from times 06.00 and 18.00.</span>
data is downloaded as netcdf on a regular 0.25x0.25 longitude/latitude grid at 1 hr resolution, on an Australian region defined as in the criteria.
 
  
/g/data/ub4/era5/netcdf/pressure/<variable>/<yr>/<files>
+
<span class="s1">The accumulations are over the hour ending at the forecast step which is a different behaviour from era interim.</span> <span class="s1">The 0 step is always zero because it would be the accumulation from timestep and timestep +0 hours (I.e. 06.00 to 06.00)</span>
  
<span class="s1">filename:</span>
+
<span class="s1">The others would be for example for precipitation:</span> <span class="s1">Time 06.00 Step 1: forecast of accumulated precipitation between 06.00-07.00</span> <span class="s1">Time 06.00 Step 19: forecast of accumulated precipitation between 06.00-01.00+1 day</span>
<variable><span class="s1">_era5_aus_<from-date>_<to-date>.nc</span>
 
  
<span class="s1">NB variable names are as in the ECMWF param tables, our ERAI netcdf files were following where possible the CMIP conventions</span>
+
&nbsp;
 
 
===='''Wave surface level'''====
 
data is downloaded as netcdf on a regular 0.5x0.5 longitude/latitude grid at 1 hr resolution, on a global grid.
 
 
 
/g/data/ub4/era5/netcdf/wave/<variable>/<yr>/<files>
 
 
 
<span class="s1">filename:</span>
 
<variable><span class="s1">_era5_global_<from-date>_<to-date>.nc</span>
 
 
 
===='''Requested variables'''====
 
 
 
[https://docs.google.com/spreadsheets/d/1CVc2xYYhgQOTN9eb5qS2HfFUa2ubuqZ0Ox6Ws3Y5KTQ/edit?usp=sharing | ERA5 variable list]
 
 
 
===Note on forecasts===
 
<span class="s1">Forecasts are accumulated values defined at 20 steps (0-19) starting from times 06.00 and 18.00.</span>
 
  
<span class="s1">The accumulations are over the hour ending at the forecast step which is a different behaviour from era interim.</span>
+
&nbsp;
<span class="s1">The 0 step is always zero because it would be the accumulation from timestep and timestep +0 hours (I.e. 06.00 to 06.00)</span>
 
  
<span class="s1">The others would be for example for precipitation:</span>
+
[[Category:Dataset]]
<span class="s1"> Time 06.00 Step 1: forecast of accumulated precipitation between 06.00-07.00</span>
 
<span class="s1"> Time 06.00 Step 19: forecast of accumulated precipitation between 06.00-01.00+1 day</span>
 

Latest revision as of 02:30, 3 May 2021

NCI is now mananging the official replica for this dataset. The data in ub4 is not anymore available, we removed access and we will delete the actual files in June.

For any information on the new collection and transition: ERA5 Community site. For any direct enquiry: help@nci.org.au .

NB. any enquiry via the CWS helpdesk will have to be transferred to the right queue and would be delayed!

ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the Copernicus Climate Change Service (C3S).

You can use the ERA5 Explorer tool to get a preview of the data (only few variables currently available). Or you can use the CDS toolbox to do some of your analysis without download the data. As descirbed on the CDS website the toolbox is "Free and available to everyone, the Toolbox links raw data to online computing power through a programming interface. In your own online workspace you can create applications in Python (a programming language) and run them on the CDS computers, allowing you to retrieve the data you are interested in, make the calculations you require and display the results in the format that suits your needs. You can download graphs, maps and data, and also share your online creations with other users."

Data processing for ERA5 is carried out by ECMWF, using ECMWFS' Earth System model IFS, cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.

For more information on ERA5 data refer to the ECMWF ERA5 documentation.

For more specific information on ERA5 Land: ECMWF ERA5 Land documentation.

ERA5 derived products

Recently ECMWF has released new ERA5 derived products. These ranged from new variables calculated from ERA5 as for the fire indices or bias -corrected, interpolated versions of ERA5 main variables themselves. For a list check the Copernicus CDS.

CEMS-Fire - Fire danger indices historical data from the Copernicus Emergency Management Service

This ERA5 derived product was recently added to the Climate Data Store. It provides complete historical reconstruction of meteorological conditions favourable to the start, spread and sustainability of fires. The fire danger metrics provided are part of a vast dataset produced by the Copernicus Emergency Managment Service for the European Forest Fire Information System (EFFIS). The European Forest Fire Information System incorporates the fire danger indices for three different models developed in Canada, United States and Australia. In this dataset the fire danger indices are calculated using weather forecast from historical simulations provided by ECMWF ERA5 reanalysis (see more on CDS website). They have a daily temporal resolution on a 0.25X0.25 degrees grid. Currently they're available for the same period as ERA5 1979 to present.

WFDE5 - Near surface meteorological variables from 1979 to 2018 derived from bias-corrected reanalysis

This dataset provides bias-corrected reconstruction of near-surface meteorological variables derived from the fifth generation of the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric reanalyses (ERA5). It is intended to be used as a meteorological forcing dataset for land surface and hydrological models (see more on the CDS website).

 

Known Issues

Known issues from ECMWF website

Issue affecting WFDE5 v1.0  "Surface downwelling shortwave radiation" data. This is resolved in v1.1.

Dec 7 2018: an error has been discovered affecting the values of the following variables

CDS Dataset ERA5 hourly data on pressure levels from 2000 to present]
Parameters affected U-component of wind, V-component of wind
Level All pressure levels
Time periods affected 01 Jan. 2000 - 31 Dec. 2008
01 Jun. 2018 - present

This problem was caused by the computation of U/V from Vorticity (Vo) and Divergence (D) already interpolated onto a regular grid in CDS. U/V should have been computed from Vo and D in the spherical harmonics representation (the native model representation) and then interpolated to a regular grid before saving to the climate data store.

The problem is now fixed and we have re-downloading the data, download should be completed by the 19 March 2019. From the CDS communication: "During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a known issue. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."

Issues and other feedback shared by users: you can check this page for issues and other information shared by other users of ERA5, feel free to add your own contribution.

License

ERA5 data is open access and free to download for all uses, including commercial use. As for ERAI and other ECMWF datasets, though, you need to register with ECMWF and accept their terms and conditions, in this particular case ERA5 is covered by the Copernicus license

 

Note on forecasts

Forecasts are accumulated values defined at 20 steps (0-19) starting from times 06.00 and 18.00.

The accumulations are over the hour ending at the forecast step which is a different behaviour from era interim. The 0 step is always zero because it would be the accumulation from timestep and timestep +0 hours (I.e. 06.00 to 06.00)

The others would be for example for precipitation: Time 06.00 Step 1: forecast of accumulated precipitation between 06.00-07.00 Time 06.00 Step 19: forecast of accumulated precipitation between 06.00-01.00+1 day