GRIB Format Files

Revision as of 00:28, 14 July 2021 by P.petrelli (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

GRIB is a data file format for climate and weather data, commonly used by institutions such as ECMWF. You may come across GRIB format files when working with data from other Centres, such as the ERA-Interim reanalysis

GRIB files can be worked with using the tools in the NCI `grib_api` module. The official documentation gives the full list of commands available, this is just a subset.

Inspecting GRIB files

To see the contents of a GRIB file use the `grib_ls` program:

$ grib_ls /g/data1/ub4/erai/grib/oper_an_pl/fullres/2014/ei_oper_an_pl_075x075_90N0E90S35925E_20140401_20140405 | head
edition      centre       typeOfLevel  level        dataDate     stepRange    dataType     shortName    packingType  gridType
1            ecmf         isobaricInhPa  1            20140401     0            an           pv           grid_simple  regular_ll
1            ecmf         isobaricInhPa  1            20140401     0            an           z            grid_simple  regular_ll
1            ecmf         isobaricInhPa  1            20140401     0            an           t            grid_simple  regular_ll

Fields in a grib file are stored as 2d horizontal slices, so for a 4d field there will be slices for each vertical level and time value.

Filtering results

You can filter the results for most GRIB commands by using the `-w}}` flag. The filter can be any of the listed columns, so for instance to only list 'u' fields you could use `{{-w shortName=u` ?'"`UNIQ--syntaxhighlight-00000002-QINU`"'?

There are many more keys you can filter by, you can use the `| grib_dump` command on a file to show the full list of keys. Some useful keys are:

shortName ECMWF field name, e.g. `u`
level Vertical level, e.g. `700` (Units will depend on the file)
cfName CF Standard Name, e.g. `eastward_wind`
dataDate Date, e.g. `20140402`
dataTime Time, e.g. `1800`

You can have multiple constraints in the same command, by separating values with '/}}' and keys with ',', e.g. '{{-w short_name=u/v/vo,level=500/700/850/925'.

Splitting and merging files

The `| grib_copy` command can be used to create a new grib file, both by combining multiple input files together as well as applying filter operations: ?'"`UNIQ--syntaxhighlight-00000003-QINU`"'?

Converting to Netcdf

The `| grib_to_netcdf}}` command will convert a GRIB file to NetCDF format. This command doesn't accept filters, so first make a copy of only the fields you need with `{{grib_copy`.

$ grib_to_netcdf -o erai_vo_201404.grib
$ ncdump -h
netcdf erai_vo_201404 {
    longitude = 480 ;
    latitude = 241 ;
    level = 4 ;
    time = 120 ;
    float longitude(longitude) ;
        longitude:units = "degrees_east" ;
        longitude:long_name = "longitude" ;
    float latitude(latitude) ;
        latitude:units = "degrees_north" ;
        latitude:long_name = "latitude" ;
    int level(level) ;
        level:units = "millibars" ;
        level:long_name = "pressure_level" ;
    int time(time) ;
        time:units = "hours since 1900-01-01 00:00:0.0" ;
        time:long_name = "time" ;
    short vo(time, level, latitude, longitude) ;
        vo:scale_factor = 2.0322676361996e-08 ;
        vo:add_offset = -9.61583905422992e-05 ;
        vo:_FillValue = -32767s ;
        vo:missing_value = -32767s ;
        vo:units = "s'''-1" ;
        vo:long_name = "Vorticity (relative)" ;
        vo:standard_name = "atmosphere_relative_vorticity" ;

// global attributes:
        :Conventions = "CF-1.0" ;
        :history = "2017-11-22 00:14:16 GMT by grib_to_netcdf-1.12.3: grib_to_netcdf -o erai_vo_201404.grib" ;

By default the data is compressed using a scale and offset, to disable this you can use `-D NC_FLOAT`.