Difference between revisions of "Data FAQ"

Line 1: Line 1:
 
  
 
=== <span style="color:#2980b9;">'''Where should I publish my data?'''</span> ===
 
=== <span style="color:#2980b9;">'''Where should I publish my data?'''</span> ===
Line 13: Line 12:
 
If you're confused feel free to ask us we are always happy to provide advice and support and whatever you choose to do remember to report the title, citation and doi on Clever. Published datasets are part of the Centre&nbsp;KPIs and something we report to our funders. It will only take a couple of minutes.
 
If you're confused feel free to ask us we are always happy to provide advice and support and whatever you choose to do remember to report the title, citation and doi on Clever. Published datasets are part of the Centre&nbsp;KPIs and something we report to our funders. It will only take a couple of minutes.
  
<span style="color:#2980b9;">'''Where should I publish my code?'''</span>
+
=== <span style="color:#2980b9;">'''Where should I publish my code?'''</span> ===
  
 
More and more frequently you are reuqired to publish also your code by journal editors. This is important to make your research reproducible, in particular for the paper reviewers. It&nbsp;can&nbsp;also be an opportunity to share a successful code with other researchers. If your code is properly published and someone else uses it, then it is easier for them to cite you as the author. Like data, code represents part of your work and funders are starting looking at all research products not only papers when reviewing applications. Putting your code on GitHub or another version control service helps to keep track of the code, expose it to others and manage pontetial issue sans enhancements. But GitHub is not ideal if you want to pinpoint the code you used for a paper or to create some data. Zenodo is a platform that integrates well with GitHub to allow you [https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiL1tLP8snvAhVL7XMBHUolAmIQFjAAegQIAhAD&url=https://guides.github.com/activities/citable-code/&usg=AOvVaw1VVBoyAZecm_pcXGudJRHP to publish automatically every release]. Zenodo will keep a snapshot in time of your code and assign a DOI to it. if you're not using GitHub you can simply upload your files directly in Zenodo. These are the reasons we started a Zenodo community for a [https://zenodo.org/communities/arc-coe-clex/?page=1&size=20 CLEX Code Collection (CCC)] in 2020. We published there some of our code and&nbsp;code used to produce papers as&nbsp;required by journal editors. We are now looking into broadening this and actively seek contributions of code, notebooks etc that might be useful to others. Zenodo has given our codes much more visibility than GitHub and some of the code has receveid lots of views and downloads.&nbsp;
 
More and more frequently you are reuqired to publish also your code by journal editors. This is important to make your research reproducible, in particular for the paper reviewers. It&nbsp;can&nbsp;also be an opportunity to share a successful code with other researchers. If your code is properly published and someone else uses it, then it is easier for them to cite you as the author. Like data, code represents part of your work and funders are starting looking at all research products not only papers when reviewing applications. Putting your code on GitHub or another version control service helps to keep track of the code, expose it to others and manage pontetial issue sans enhancements. But GitHub is not ideal if you want to pinpoint the code you used for a paper or to create some data. Zenodo is a platform that integrates well with GitHub to allow you [https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiL1tLP8snvAhVL7XMBHUolAmIQFjAAegQIAhAD&url=https://guides.github.com/activities/citable-code/&usg=AOvVaw1VVBoyAZecm_pcXGudJRHP to publish automatically every release]. Zenodo will keep a snapshot in time of your code and assign a DOI to it. if you're not using GitHub you can simply upload your files directly in Zenodo. These are the reasons we started a Zenodo community for a [https://zenodo.org/communities/arc-coe-clex/?page=1&size=20 CLEX Code Collection (CCC)] in 2020. We published there some of our code and&nbsp;code used to produce papers as&nbsp;required by journal editors. We are now looking into broadening this and actively seek contributions of code, notebooks etc that might be useful to others. Zenodo has given our codes much more visibility than GitHub and some of the code has receveid lots of views and downloads.&nbsp;

Revision as of 18:16, 24 March 2021

Where should I publish my data?

You usually have at least three options and there's not a straight answer it depends on what you're publishing and why.

CLEX Data Collection on NCI: this is the best place if you have actually produced the data yourself. We'll help you document your data and make it user friendly, it will be part of a climate data collection and so it will be easier to discover. NCI also has more storage capacity than other repositories and services which are designed around the netcdf format.

Institutional repository: this can be adeguate if you have a small dataset and it is really specific to your study or only a subset/post processing of another dataset. While institutions offer some data curation they usually won't check the data is well described, consistent and user friendly, so you will get a DOI for your record but no added value.

Zenodo, Figshare, Mendeley: these services are free and you can create your own account, create a record for your data and get a DOI fairly easily and quickly. You can also publish here different kind of materials. This can be useful if you want to publish some very specific data, for example code and data to produce a specific figure required to publish a paper. If you are publishing data that can be actually useful to others, then you want to make sure is FAIR (see above for a full description). Findable, which is harder when your record is not part of a discipline repository or collection or you haven't used keywords in an effective manner. Accessible, Interoperable and Reusable which means the data should have enough metadata, use standards, be properly described. Finally the data size is limited to 50 GB and you won't get any additional data services apart from HTTP download. This should be your last resort, still if you go this way please make sure you document your data properly. we are happy to provide support and review your record. We are also looking into creating  CLEX community in Zenodo to be able to collect in one place these data records.

If you're confused feel free to ask us we are always happy to provide advice and support and whatever you choose to do remember to report the title, citation and doi on Clever. Published datasets are part of the Centre KPIs and something we report to our funders. It will only take a couple of minutes.

Where should I publish my code?

More and more frequently you are reuqired to publish also your code by journal editors. This is important to make your research reproducible, in particular for the paper reviewers. It can also be an opportunity to share a successful code with other researchers. If your code is properly published and someone else uses it, then it is easier for them to cite you as the author. Like data, code represents part of your work and funders are starting looking at all research products not only papers when reviewing applications. Putting your code on GitHub or another version control service helps to keep track of the code, expose it to others and manage pontetial issue sans enhancements. But GitHub is not ideal if you want to pinpoint the code you used for a paper or to create some data. Zenodo is a platform that integrates well with GitHub to allow you to publish automatically every release. Zenodo will keep a snapshot in time of your code and assign a DOI to it. if you're not using GitHub you can simply upload your files directly in Zenodo. These are the reasons we started a Zenodo community for a CLEX Code Collection (CCC) in 2020. We published there some of our code and code used to produce papers as required by journal editors. We are now looking into broadening this and actively seek contributions of code, notebooks etc that might be useful to others. Zenodo has given our codes much more visibility than GitHub and some of the code has receveid lots of views and downloads. 

Do CMIP5/CMIP6 variables coming from the same simulations have the same version number?

For CMIP5 there's no way to be 100% sure that two versions come out of the same simulation. The versioning instructions where quite unclear and interpreted differently by different modelling groups. In the last couple of years, it occurred to few groups (for example GFDL) to add a "simulation_id" to their attributes but this is the exception rather than the norm. I'd like to assume that same version means the same simulation, but having a different version really just means that the group of variables has been published later, or maybe that one of them was calculated wrongly in the post-processing and has been re-published under a new version. A completely different simulation with a different configuration, initialization etc should have a different ensemble code so anything with r1i1p1, for example, should come from the same run, even if part of it or its post-processing might have been updated. More information might be available directly from the modelling groups. 

In CMIP6 again the way versions ar eapllied is incnsistent but you can now use the tracking_ids to actually trace a file provenance, ESDOC gives you full information on model simulations and each file, simualtion, model, experiment has an handle you can use to track down information.

Using the CLEx Roadmap data management tool is not useful for me as I don't use NCI servers.

Using Roadmap is really independent of NCI, while it contains some specific information on NCI systems since they are common to many in the Centre, it is more about data management in general, regardless which hardware you use. Even if you are using your laptop it is good to have a data workflow, a plan of what you will be doing at different stages of your research. For example, to publish an RDA record for your data with us, you would now fill one of these plans, and the advantage would be that you can easily export that as a document which you can always adapt and re-use.

DMP will be compulsory for universities and ARC grants, publishing your data is already compulsory for most journals. Plus, at CMS we really want to hear from users that are not using NCI, users we don't normally hear from. So we get a better idea of what everybody in the Centre is doing and we can create new training resources or support all our users in a better way.