Is there a standard format for sharing data?

Question

We are writing a paper, and want to share the raw data. My advisor suggests that I share the data as an Excel file, but I believe that a plain text file (such as a text-delimited or comma-delimited file) would be better.

1. Is there a standard format for sharing data?
2. How do I convince my advisor that a plain text file is a better format for sharing data?

Answer 1

1. Consider non-proprietary format as a good practice

You might find an argument in the Tim Berners-Lee 5 star approach. When discussing Open Data (something we should be embracing more in academia as well), he presents the following:

Under the star scheme, you get one (big!) star if the information has been made public at all, even if it is a photo of a scan of a fax of a table -- if it has an open licence. The you get more stars as you make it progressively more powerful, easier for people to use.

★ Available on the web (whatever format) but with an open licence, to be Open Data

★★ Available as machine-readable structured data (e.g. excel instead of image scan of a table)

★★★ as (2) plus non-proprietary format (e.g. CSV instead of excel)

★★★★ All the above plus, Use open standards from W3C (RDF and SPARQL) to identify things, so that people can point at your stuff

★★★★★ All the above, plus: Link your data to other people’s data to provide context

It is best to publish in an non-proprietary format (csv would be fine) as you claimed. Stars 4 & 5 are for Linked Data structure which is a nice thing to aim.

2. Publish in a reliable repository thinking long-term preservation

Citation and versioning are very important if you want to alter something on your data-sets in the future. I would recommend you publish your data in Figshare. Research made publicly available of figshare gets allocated a DataCite DOI at point of publication. It supports versioning as well.

Another alternative for a repository is DataVerse suggested by Thomas below

Answer 2

No.

Depends greatly on the field and kind of data.

Some fields do have relatively standard formats:

• In Astronomy, FITS is considered pretty much the standard for imagery.
• In machine learning, CSV/TSV is relatively common -- though as the data set sizes get larger and richer, other formats come into play. You can troll the UCI ML Repository to get a feeling for what's common.

In general though, I'd argue that plain-text based formats are what you want to preserve. If, happiness of happiness, your paper becomes hugely influential and you only provide an .xls file, you can bet the first thing that's going to happen is a bunch of ersatz plain-text copies will appear.

If you want to annotate your data, and that's what is swaying your advisor to Excel, perhaps consider something like XML or JSON -- easy for the computer to process and plain text based, yet free-form enough that you can annotate it however you'd like.

Answer 3

I agree with everything Matthew G. has said. But I wanted to offer some additional thoughts on your second point about convincing your advisor to use a plain-text file format. Demonstrate any of the following:

1. Send the Excel file to a friend running Linux and ask them to open it.
2. Save the Excel file in Excel 2007+ format and try to open it in an Excel version from 2003 or earlier.
3. Try to open the Excel file in a text editor to make changes.

Then repeat all of the above with a CSV/TSV. The portability of the plain-text format will be shown to be unmatched.