Difference between revisions of "ERA5"

m
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
 +
'''<span style="color:#c0392b;">NCI is now mananging the official replica for this dataset. The data in ub4 is not anymore available, we removed access and we will delete the actual files in June.</span>'''
 +
 +
'''<span style="color:#c0392b;">For any information on the new collection and transition:&nbsp;</span><span style="color:#c0392b;">[https://opus.nci.org.au/display/ERA5/ERA5+Community+Home ERA5 Community site].</span><span style="color:#c0392b;">&nbsp;For any direct enquiry: help@nci.org.au .</span>'''
 +
 +
'''<span style="color:#c0392b;">NB. any enquiry via the CWS helpdesk will have to be transferred to the right queue&nbsp;and would be delayed!</span>'''
  
 
ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the [http://climate.copernicus.eu/ Copernicus Climate Change Service (C3S)].
 
ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the [http://climate.copernicus.eu/ Copernicus Climate Change Service (C3S)].
 +
 +
You can use the [https://cds.climate.copernicus.eu/cdsapp#!/software/app-era5-explorer?tab=app ERA5 Explorer] tool to get a preview of the data (only few variables currently available). Or you can use the [https://cds.climate.copernicus.eu/toolbox/doc/index.html# CDS toolbox] to do some of your analysis without download the data. As descirbed on the CDS website the toolbox is&nbsp;"Free and available to everyone, the Toolbox links raw data to online computing power through a programming interface. In your own online workspace you can create applications in Python (a programming language) and run them on the CDS computers, allowing you to retrieve the data you are interested in, make the calculations you require and display the results in the format that suits your needs. You can download graphs, maps and data, and also share your online creations with other users."
  
 
Data processing for ERA5 is carried out by [http://www.ecmwf.int/ ECMWF], using ECMWFS' [https://www.ecmwf.int/en/research/modelling-and-prediction Earth System model IFS], cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.
 
Data processing for ERA5 is carried out by [http://www.ecmwf.int/ ECMWF], using ECMWFS' [https://www.ecmwf.int/en/research/modelling-and-prediction Earth System model IFS], cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.
  
We are providing a small subset of ERA5 while waiting for a longer term solution to store and manage this big dataset. Currently data is available from 1979 to present. We have covered the period 2000-2018 for all the requested variables and we are in the process of downloading the other years, refer to the ERA5&nbsp;variable list for more details. ERA5 comprises different data stream, we have only a small subset of the surface operative analysis. For the moment we are not considering downloading model or pressure level data as the size of these are quite big and we wouldn't be able to provide enough data to use them to run a model. For more information on ERA5 data refer to the [https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation ECMWF website].
+
For more information on ERA5 data refer to the [https://confluence.ecmwf.int/display/CKB/ERA5+data+documentation ECMWF ERA5 documentation].
  
If you need other variables for your current research then you can contact us at climate_help@nci.org.au
+
<span class="s1">For more specific information on ERA5 Land: [https://confluence.ecmwf.int/display/CKB/ERA5-Land:+data+documentation ECMWF ERA5 Land documentation].</span>
  
'''NB''': this is a temporary fix so we will download only variables which are useful for a variety of projects and that are needed now, if you can use ERA-Interim instead or you will be using ERA5 in the future but not in the next few months, then we won't consider your request.
+
=== '''ERA5 derived products''' ===
  
&nbsp;
+
Recently ECMWF has released new ERA5&nbsp;derived products.&nbsp;These ranged from new variables&nbsp;calculated from ERA5 as for the fire indices or bias -corrected, interpolated versions of ERA5 main variables themselves.&nbsp;For a list check the [https://cds.climate.copernicus.eu/cdsapp#!/search?text=ERA5 Copernicus CDS].
  
=== '''Requests constraints''' ===
+
==== '''CEMS-Fire - Fire danger indices historical data from the Copernicus Emergency Management Service''' ====
  
<span class="s1">We will only accept requests for:</span> <span class="s1">- data on regular 0.25 by 0.25 lat/lon grid</span> <span class="s1">- global region for surface variables </span> <span class="s1">- Austral-Asian region for pressure level. The region includes Australia and the Maritime continent and the CORDEX domain and is defined as:</span> <span class="s1">Latitude: 20 to -57</span> <span class="s1">Longitude: 78 to 220</span> <span class="s1">NB longitude for the global region is -180 to 180</span> <span class="s1">- analysis data and no ensemble data (analysis includes surface forecast fields such as precipitation). It is currently impossible to provide input data for a climate model</span> <span class="s1">You can submit a request for downloading following these criteria. We will only consider requests that specify the exact code/stream etc. You can refer for more information to the documentation here and to the <span class="s3">[https://protect-au.mimecast.com/s/-dp6CZYM5DFXmX6BIzaRdX?domain=apps.ecmwf.int ERA5 catalogue]</span>. Be as specific as possible in your request: grib or netcdf format? which forecast steps do you need if you request forecast data? (see ERA5 website)</span>
+
This ERA5 derived product was recently added to the Climate Data Store. It&nbsp;provides complete historical reconstruction of meteorological conditions favourable to the start, spread and sustainability of fires. The fire danger metrics provided are part of a vast dataset produced by the Copernicus Emergency Managment Service for the European Forest Fire Information System (EFFIS). The European Forest Fire Information System incorporates the fire danger indices for three different models developed in Canada, United States and Australia. In this dataset the fire danger indices are calculated using weather forecast from historical simulations provided by ECMWF ERA5 reanalysis&nbsp;(see more on [https://cds.climate.copernicus.eu/cdsapp#!/dataset/cems-fire-historical?tab=overview CDS website]). They have a daily temporal resolution on a&nbsp;0.25X0.25 degrees grid. Currently they're available for the same period as ERA5 1979 to present.
  
&nbsp;
+
==== '''WFDE5 - Near surface meteorological variables from 1979 to 2018 derived from bias-corrected reanalysis''' ====
  
=== '''Latest updates''' ===
+
This dataset provides bias-corrected reconstruction of near-surface meteorological variables derived from the fifth generation of the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric reanalyses (ERA5). It is intended to be used as a meteorological forcing dataset for land surface and hydrological models&nbsp;(see&nbsp;more on the [https://cds.climate.copernicus.eu/cdsapp#!/dataset/derived-near-surface-meteorological-variables?tab=overview CDS website]).
  
We added minimum and maximum 2m temperature, these surface variables are the first ones available for the entire period 1979 -2018 together with: mean sea level pressure,&nbsp;2m temperature, convective precipitation and total precipitation.
+
&nbsp;
  
=== '''Known Issues''' ===
+
'''Known Issues'''
  
 
[https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation#ERA5datadocumentation-Knownissues Known issues from ECMWF website]
 
[https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation#ERA5datadocumentation-Knownissues Known issues from ECMWF website]
 +
 +
Issue affecting WFDE5 v1.0 &nbsp;"Surface downwelling shortwave radiation" data.&nbsp;This is resolved in v1.1.
  
 
Dec 7 2018: an error has been discovered affecting the values of the following variables
 
Dec 7 2018: an error has been discovered affecting the values of the following variables
Line 46: Line 56:
  
 
The problem is now fixed and we have re-downloading&nbsp;the data, download should be completed by the 19 March 2019. From the CDS communication: "<span class="s1">During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a <span class="s2">[https://protect-au.mimecast.com/s/vf55CQnzVqt4vDL7hxh02-?domain=confluence.ecmwf.int known issue]</span>. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."</span>
 
The problem is now fixed and we have re-downloading&nbsp;the data, download should be completed by the 19 March 2019. From the CDS communication: "<span class="s1">During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a <span class="s2">[https://protect-au.mimecast.com/s/vf55CQnzVqt4vDL7hxh02-?domain=confluence.ecmwf.int known issue]</span>. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."</span>
 +
 +
<span class="s1">[[Era5_issues|Issues and other feedback shared by users]]: you can check this page for issues and other information shared by other users of ERA5, feel free to add your own contribution.</span>
  
 
=== '''License''' ===
 
=== '''License''' ===
  
<span style="background-color: #ffffff; color: #172b4d; font-family: -apple-system,system-ui,'Segoe UI',Roboto,Oxygen,Ubuntu,'Fira Sans','Droid Sans','Helvetica Neue',sans-serif; font-size: 14px;">ERA5 data is open access and free to download for all uses, including commercial use.</span> As for ERAI and other ECMWF datasets, though, you need to have an [https://apps.ecmwf.int/registration/ account with ECMWF] and accept their terms and conditions, in this particular case ERA5 is covered by the [http://apps.ecmwf.int/datasets/licences/copernicus/ Copernicus license]
+
<span style="background-color: #ffffff; color: #172b4d; font-family: -apple-system,system-ui,'Segoe UI',Roboto,Oxygen,Ubuntu,'Fira Sans','Droid Sans','Helvetica Neue',sans-serif; font-size: 14px;">ERA5 data is open access and free to download for all uses, including commercial use.</span> As for ERAI and other ECMWF datasets, though, you need to&nbsp;[https://apps.ecmwf.int/registration/ register&nbsp;with ECMWF] and accept their terms and conditions, in this particular case ERA5 is covered by the [http://apps.ecmwf.int/datasets/licences/copernicus/ Copernicus license]
 
 
=== '''ERA5 on raijin''' ===
 
 
 
==== '''Surface''' ====
 
 
 
data is downloaded as netcdf on a regular 0.25x0.25 longitude/latitude global grid at 1 hr resolution, as available from the Copernicus server. These are the maximum spatial and temporal resolutions possible for this dataset.
 
 
 
/g/data/ub4/era5/netcdf/surface/<variable>/<yr>/<files>
 
 
 
<span class="s1">filename:</span> <variable><span class="s1">_era5_global_<from-date>_<to-date>.nc</span>
 
 
 
==== '''Pressure levels''' ====
 
 
 
data is downloaded as netcdf on a regular 0.25x0.25 longitude/latitude grid at 1 hr resolution, on an Australian region defined as in the criteria.
 
 
 
/g/data/ub4/era5/netcdf/pressure/<variable>/<yr>/<files>
 
 
 
<span class="s1">filename:</span> <variable><span class="s1">_era5_aus_<from-date>_<to-date>.nc</span>
 
 
 
<span class="s1">NB variable names are as in the ECMWF param tables, our ERAI netcdf files were following where possible the CMIP conventions</span>
 
 
 
==== '''Wave surface level''' ====
 
 
 
data is downloaded as netcdf on a regular 0.5x0.5 longitude/latitude grid at 1 hr resolution, on a global grid.
 
 
 
/g/data/ub4/era5/netcdf/wave/<variable>/<yr>/<files>
 
 
 
<span class="s1">filename:</span> <variable><span class="s1">_era5_global_<from-date>_<to-date>.nc</span>
 
 
 
==== '''Requested variables''' ====
 
  
[https://docs.google.com/spreadsheets/d/1CVc2xYYhgQOTN9eb5qS2HfFUa2ubuqZ0Ox6Ws3Y5KTQ/edit?usp=sharing ERA5 variable list]
+
=== &nbsp; ===
 
 
&nbsp;
 
  
 
==== '''Note on forecasts''' ====
 
==== '''Note on forecasts''' ====
Line 95: Line 75:
 
&nbsp;
 
&nbsp;
  
[[Category:Dataset]][[Category:Clex-managed-data]]
+
&nbsp;
 +
 
 +
[[Category:Dataset]]

Latest revision as of 01:30, 3 May 2021

NCI is now mananging the official replica for this dataset. The data in ub4 is not anymore available, we removed access and we will delete the actual files in June.

For any information on the new collection and transition: ERA5 Community site. For any direct enquiry: help@nci.org.au .

NB. any enquiry via the CWS helpdesk will have to be transferred to the right queue and would be delayed!

ERA5 is a climate reanalysis dataset, covering the period 1950 to present. ERA5 is being developed through the Copernicus Climate Change Service (C3S).

You can use the ERA5 Explorer tool to get a preview of the data (only few variables currently available). Or you can use the CDS toolbox to do some of your analysis without download the data. As descirbed on the CDS website the toolbox is "Free and available to everyone, the Toolbox links raw data to online computing power through a programming interface. In your own online workspace you can create applications in Python (a programming language) and run them on the CDS computers, allowing you to retrieve the data you are interested in, make the calculations you require and display the results in the format that suits your needs. You can download graphs, maps and data, and also share your online creations with other users."

Data processing for ERA5 is carried out by ECMWF, using ECMWFS' Earth System model IFS, cycle 41r2. The name ERA refers to 'ECMWF ReAnalysis', with ERA5 being the fifth major global reanalysis produced by ECMWF (after FGGE, ERA-15, ERA-40, ERA-Interim). The next reanalysis, ERA6, is planned for around 2020.

For more information on ERA5 data refer to the ECMWF ERA5 documentation.

For more specific information on ERA5 Land: ECMWF ERA5 Land documentation.

ERA5 derived products

Recently ECMWF has released new ERA5 derived products. These ranged from new variables calculated from ERA5 as for the fire indices or bias -corrected, interpolated versions of ERA5 main variables themselves. For a list check the Copernicus CDS.

CEMS-Fire - Fire danger indices historical data from the Copernicus Emergency Management Service

This ERA5 derived product was recently added to the Climate Data Store. It provides complete historical reconstruction of meteorological conditions favourable to the start, spread and sustainability of fires. The fire danger metrics provided are part of a vast dataset produced by the Copernicus Emergency Managment Service for the European Forest Fire Information System (EFFIS). The European Forest Fire Information System incorporates the fire danger indices for three different models developed in Canada, United States and Australia. In this dataset the fire danger indices are calculated using weather forecast from historical simulations provided by ECMWF ERA5 reanalysis (see more on CDS website). They have a daily temporal resolution on a 0.25X0.25 degrees grid. Currently they're available for the same period as ERA5 1979 to present.

WFDE5 - Near surface meteorological variables from 1979 to 2018 derived from bias-corrected reanalysis

This dataset provides bias-corrected reconstruction of near-surface meteorological variables derived from the fifth generation of the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric reanalyses (ERA5). It is intended to be used as a meteorological forcing dataset for land surface and hydrological models (see more on the CDS website).

 

Known Issues

Known issues from ECMWF website

Issue affecting WFDE5 v1.0  "Surface downwelling shortwave radiation" data. This is resolved in v1.1.

Dec 7 2018: an error has been discovered affecting the values of the following variables

CDS Dataset ERA5 hourly data on pressure levels from 2000 to present]
Parameters affected U-component of wind, V-component of wind
Level All pressure levels
Time periods affected 01 Jan. 2000 - 31 Dec. 2008
01 Jun. 2018 - present

This problem was caused by the computation of U/V from Vorticity (Vo) and Divergence (D) already interpolated onto a regular grid in CDS. U/V should have been computed from Vo and D in the spherical harmonics representation (the native model representation) and then interpolated to a regular grid before saving to the climate data store.

The problem is now fixed and we have re-downloading the data, download should be completed by the 19 March 2019. From the CDS communication: "During our investigations, we found that wind values are far too low on pressure levels at the poles in CDS, which has been documented as a known issue. Please use winds at neighbouring locations at 89.75 (N/S) and 89.5 (N/S) for Reanalysis and Ensemble members winds instead of 90 (N/S)."

Issues and other feedback shared by users: you can check this page for issues and other information shared by other users of ERA5, feel free to add your own contribution.

License

ERA5 data is open access and free to download for all uses, including commercial use. As for ERAI and other ECMWF datasets, though, you need to register with ECMWF and accept their terms and conditions, in this particular case ERA5 is covered by the Copernicus license

 

Note on forecasts

Forecasts are accumulated values defined at 20 steps (0-19) starting from times 06.00 and 18.00.

The accumulations are over the hour ending at the forecast step which is a different behaviour from era interim. The 0 step is always zero because it would be the accumulation from timestep and timestep +0 hours (I.e. 06.00 to 06.00)

The others would be for example for precipitation: Time 06.00 Step 1: forecast of accumulated precipitation between 06.00-07.00 Time 06.00 Step 19: forecast of accumulated precipitation between 06.00-01.00+1 day