Difference between revisions of "CF checker"

(Created page with "There are a few options when it comes to check if your files are CF compliant. You can use online checkers as these: [https://compliance.ioos.us/index.html https://compliance....")
 
Line 1: Line 1:
There are a few options when it comes to check if your files are CF compliant. You can use online checkers as these: [https://compliance.ioos.us/index.html https://compliance.ioos.us/index.html] [http://cfconventions.org/compliance-checker.html http://cfconventions.org/compliance-checker.html] You upload yur netcdf file on the web and you get back a report.   This is fine if you have one small file but if you want to check a lot of files you have to use a checker on oyur computer. We installed the [https://github.com/ioos/compliance-checker ioos checker] in our conda environments. The ioos checker is python based and it is the same checker used by NCI on the data they publish. Using it is very simple, nust load the module   '''     module use /g/data3/hh5/public/modules''' '''     module load conda/analysis3''' To can now call the checker cchecker.py  cchecker.py -t=acdd -f json_new -o tpca_acdd.json -c strict /g/data/ua8/CLEX_Data/TPCA_reprocessing/tmp/*/tpca_*_2007.nc  
+
 
 +
There are a few options when it comes to check if your files are CF compliant. You can use online checkers as these:
 +
 
 +
[https://compliance.ioos.us/index.html https://compliance.ioos.us/index.html] [http://cfconventions.org/compliance-checker.html http://cfconventions.org/compliance-checker.html]
 +
 
 +
You upload yur netcdf file on the web and you get back a report.
 +
 
 +
This is fine if you have one small file but if you want to check a lot of files you have to use a checker on yuor computer. We installed the [https://github.com/ioos/compliance-checker ioos checker] in our conda environments. The ioos checker is python based, as well as checking for CF conventions it works also with the ACDD conventions. NCI uses this checker and in fact they require both conventions to be applied to the data they publish.
 +
 
 +
Using it is very simple, just load the module
 +
 
 +
  '''     module use /g/data3/hh5/public/modules'''
 +
 
 +
'''       module load conda/analysis3'''
 +
 
 +
You can now call the checker on a netcdf file:
 +
 
 +
       
 +
 
 +
$ cchecker.py test_file.nc&nbsp;<br/> Running Compliance Checker on the datasets from: ['test_file.nc']<br/> --------------------------------------------------------------------------------<br/> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;IOOS Compliance Checker Report &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;<br/> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; acdd:1.3 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<br/> http://wiki.esipfed.org/index.php?title=Category:Attribute_Conventions_Dataset_Discovery<br/> --------------------------------------------------------------------------------<br/> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Corrective Actions &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;<br/> test_file.nc has 4 potential issues
 +
 
 +
<br/> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Highly Recommended &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;<br/> --------------------------------------------------------------------------------<br/> Global Attributes<br/> * Conventions does not contain 'ACDD-1.3'<br/> * summary not present<br/> .........
 +
 
 +
As you can see from the example without passing any option the tool checks for ACDD 1.3 (the latest version) compliance. The report is printed out to screen.
 +
 
 +
cchecker --help
 +
 
 +
will list all the available options, the main ones are:
 +
 
 +
-t/--test&nbsp; to choose the test;
 +
 
 +
&nbsp; &nbsp; &nbsp;ie.e -t=cf&nbsp; will test the file against&nbsp;the latest available verison of the CF conventions.
 +
 
 +
-c /-- criteria set the level to which run the test;
 +
 
 +
&nbsp; &nbsp; &nbsp; possible options are < lenient, normal, strict >, default to <normal>
 +
 
 +
-f/--format&nbsp; the output format;
 +
 
 +
&nbsp; &nbsp; &nbsp; possible options are < text, html, json, json_new >
 +
 
 +
-o/--output optional file/s to redirect output to.
 +
 
 +
You can also run the checker on severla files at one time. For example if&nbsp;
 +
 
 +
cchecker.py -t=cf -c strict -o cf_test.txt test_data/test*.nc&nbsp;

Revision as of 17:14, 11 November 2019

There are a few options when it comes to check if your files are CF compliant. You can use online checkers as these:

https://compliance.ioos.us/index.html http://cfconventions.org/compliance-checker.html

You upload yur netcdf file on the web and you get back a report.

This is fine if you have one small file but if you want to check a lot of files you have to use a checker on yuor computer. We installed the ioos checker in our conda environments. The ioos checker is python based, as well as checking for CF conventions it works also with the ACDD conventions. NCI uses this checker and in fact they require both conventions to be applied to the data they publish.

Using it is very simple, just load the module

       module use /g/data3/hh5/public/modules

       module load conda/analysis3

You can now call the checker on a netcdf file:

       

$ cchecker.py test_file.nc 
Running Compliance Checker on the datasets from: ['test_file.nc']
--------------------------------------------------------------------------------
                         IOOS Compliance Checker Report                         
                                    acdd:1.3                                    
http://wiki.esipfed.org/index.php?title=Category:Attribute_Conventions_Dataset_Discovery
--------------------------------------------------------------------------------
                               Corrective Actions                               
test_file.nc has 4 potential issues


                               Highly Recommended                               
--------------------------------------------------------------------------------
Global Attributes
* Conventions does not contain 'ACDD-1.3'
* summary not present
.........

As you can see from the example without passing any option the tool checks for ACDD 1.3 (the latest version) compliance. The report is printed out to screen.

cchecker --help

will list all the available options, the main ones are:

-t/--test  to choose the test;

     ie.e -t=cf  will test the file against the latest available verison of the CF conventions.

-c /-- criteria set the level to which run the test;

      possible options are < lenient, normal, strict >, default to <normal>

-f/--format  the output format;

      possible options are < text, html, json, json_new >

-o/--output optional file/s to redirect output to.

You can also run the checker on severla files at one time. For example if 

cchecker.py -t=cf -c strict -o cf_test.txt test_data/test*.nc