Archiving Data

Saving to MDSS

The NCI MDSS system is available for long-term storage of data.

Files saved to tape should be compressed, and small files should be gathered together using tar

mdss put

The mdss put command copies files directly to tape storage. This should only be used for files larger than 1 MB, for smaller files first collect them together with tar

For more info run man mdss

netmv / netcp

The netmv and netcp commands create a copyq job to transfer multiple files

Files can be automatically tarred and compressed as part of the copy process

NOTE: The process of compressing can use a lot of storage on /short if you're moving lots of data!

For more info run man netmv


mdssdiff syncs a directory tree to MDSS

mdssdiff is part of the CMS Conda environment

For more info see Github

Saving to University Archives

Individual Universities also have their own archives where you can store your data, for more information contact your Library or IT department