70

In an upcoming publication, I need to link to the data I used for the publication so that others can see/use the data as well — both for reviewing the given work and also for intended use in the future. However, my institution has not offered any hosting solutions and I have not (yet) found any acceptable external solutions which absolve me of financial and legal responsibility for maintaining the data and the hosting infrastructure. I will not be at the given institution for very much longer, so e.g. putting it on my personal site at the institution is not a solution. The primary data in question is about 12GB in size, so it needs to be a proper "repository" for the data rather than just e.g. an attachment distributed with the publication itself.

Nevertheless, I need to at least have a stable link to some place where the data can be located; The stability of the actual location is not as important as the stability of the link itself. How/where can I procure a permanent URL to link to research data in a publication which does not cost me anything as an individual?

Improve this question
14
  • 5
    How much data? At least in chemistry it is almost always possible to include supporting information file hosted by the publisher (if it's not too much) or there are special archives for certain type of data you can use (an example would be the ccdc)
    – user64845
    Commented Dec 23, 2017 at 13:49
  • 3
    About 12GB, so I doubt that that would be a possible avenue...
    – errantlinguist
    Commented Dec 23, 2017 at 13:53
  • 3
    I recently discovered that my institute offers the use of a file hosting service that is maintained by a network of institutions for exactly such purposes. I had never heard about it. I suggest you talk to the helpful people at your institute's library.
    – user9482
    Commented Dec 23, 2017 at 18:09
  • 4
    Is it possible to register for a doi in cases such as this?
    – Jim Belk
    Commented Dec 23, 2017 at 19:24
  • 7
    Regardless of how you choose to host the data I recommend that the publication itself contains a cryptographic hash of the data file (I am assuming it will be distributed as an archive file such that a single hash will cover it all). That will allow anybody who wants to inspect the data to verify that they have the correct data and it can also help a bit in tracking down the data should the original download link stop working.
    – kasperd
    Commented Dec 24, 2017 at 12:30

9 Answers 9

Reset to default
71

Maybe Zenodo or other "Academic Data Repository". Googling this would give you a list. Zenodo have some advantages.

  1. Gives you a DOI, Digial Object Identifier, a unique link and a academic standard for citations.
  2. You don't need acceptance to publish your data.
  3. Is a official EU project, used for giving research grants in Open AIRE project.
  4. Is hosted by CERN.
  5. Runs free software in the entire stack.
Improve this answer
5
51

If you also have some code associated with this data that you might like to share, another option might be GitHub. You wouldn't host the 12GB dataset in a GitHub repository itself; instead you would host your code, and create a readme.md file (GitHub will do this virtually automatically for you) where you write out instructions or other narrative. This is where you would include a link to wherever you've chosen to host the data. You can then update this link any time you want or need (for example, if you change institutions).

This has a number of advantages over simply finding a static place to stick the data and sharing that link:

  1. GitHub is almost a decade old and has over 20 million users, so it's not going anywhere
  2. Public repositories are free
  3. Including any code you want to share in the same place is very convenient
  4. The readme.md lets you write out whatever message you would like a future user to encounter, such as guidance not included in the original paper, errata, etc.
  5. Everything is updatable by you at any time, but still maintains the static link
  6. Using version control on your code is a fantastic habit to form
  7. GitHub makes it very easy to include copyright and licensing info
  8. You can use GitHub to build an entire website if you want to go that route (GitHub Pages), which can include what you've shared
Improve this answer
15
  • 8
    This doesn't answer the question at all. Your answer explains where to host code (OP didn't ask for that) and doesn't resolve the question about where to host the 12GB of actual data.
    – user64845
    Commented Dec 23, 2017 at 18:15
  • 47
    @DSVA It most certainly answers the question. He didn't ask where you host data, he asked how to get a static link to data. GitHub does that and more, which is what I wrote.
    – Jeff
    Commented Dec 23, 2017 at 18:17
  • 4
    @DSVA this is not a "bad" (downvote-worthy) answer even it's not amazingly "good": I have seen people doing something similar, where they e.g. create a bare-bones GitHub repo with a few example files and a note saying "This data was used for Stark et al. 2017. 'Sustainable mining of dragonglass in coastal regions'. Westerosi Geology, pp. 12--44. Contact [email protected] for the entire dataset." The only bad part about this is that I can't have this data under a personal GitHub account and don't want to create an orphan account which no one at the department uses.
    – errantlinguist
    Commented Dec 23, 2017 at 18:24
  • 11
    +1 this is a great answer. Github will provide a link as close to permanent as you are likely to find anywhere on the Internet. This way you can put the 12GB of data anywhere you like, even on multiple free hosts, as many as you can find within reason. And provide a list of links on the Github readme. If a few links die (a site like pingdom will monitor them for you, for free) you can always top the list up by uploading the data to some new hosts
    – Darren H
    Commented Dec 23, 2017 at 19:30
  • 3
    Tell us about GitHub in 50 years, then we'll talk.
    – einpoklum
    Commented Dec 24, 2017 at 17:25
30

There are services that provide enough to support 12 GB of data. For example, Figshare provides 20 GB of free space (file size limit 5 GB) for private storage and apparently unlimited public space. They state they can support larger files but not through user upload.

When you publish data you can assign a doi to the data set (this can actually be done much earlier in the process as a reserved number). Many journals also use Figshare (and likely other services) for their "Supporting information" as well. I do not know if adding such information is associated with costs.

I am only familiar (not associated) with Figshare and do not know limitations of other similar services so see this as an example. Also look in to the possibility to add the data as supporting information to your article.

Improve this answer
6
  • I see that figshare has been around for 6-7 years, which is a relatively long time on the Internet and a good sign since longevity is key here.
    – Luke Sawczak
    Commented Dec 23, 2017 at 17:50
  • 13
    I agree that the "right way" to link to a dataset is to assign a DOI to it. To that end, zenodo.org is a free service that accept up to 50GB per dataset.
    – LCT
    Commented Dec 23, 2017 at 22:37
  • @LCT There's something to that but I think people tend to overemphasize the importance of a DOI. Don't get me wrong, it does have clear advantages, in the sense of being intended to be permanent, being a standard that academics are familiar with, and being compatible (in some sense) with existing citation formats, but let's not get carried away thinking that, say, anything without a DOI is necessarily inferior.
    – David Z
    Commented Dec 24, 2017 at 0:21
  • 1
    @DavidZ An archive without DOIs isn't necessarily inferior. An archive without a reasonably well-established method for permanent document identification is, though.
    – E.P.
    Commented Dec 24, 2017 at 10:38
  • @E.P. Right, I'm just calling out the implication (intended or not) that a DOI is the "right way" to permanently identify a resource and any other type of permanent identifier is the "wrong way".
    – David Z
    Commented Dec 24, 2017 at 10:47
10

If your data is a collection of books, audio, or video files, you may host them on the Internet Archive's website, https://archive.org (upload page: https://archive.org/create/).

The Internet Archive is a San Francisco–based nonprofit digital library with the stated mission of "universal access to all knowledge." It provides free public access to collections of digitized materials, including websites, software applications/games, music, movies/videos, moving images, and nearly three million public-domain books. As of October 2016, its collection topped 15 petabytes. In addition to its archiving function, the Archive is an activist organization, advocating for a free and open Internet. [...] Founded by Brewster Kahle in May 1996.

It's free to upload and download.

Examples:

Improve this answer
1
  • 2
    The Internet Archive is also working on establishing a copy of their entire archive in Canada, as well as that they (per that official blog post) have partial copies in Egypt and the Netherlands, along with their primary archive storage in the USA. This should help provide both geographical and to some extent political diversification.
    – user
    Commented Dec 26, 2017 at 20:58
8

You could use a service that provides PURLs (persistent URLs).

Such a URL redirects to a target URL of your choice, and you can update the target URL in case you need to move to a new hosting location.

Examples

  • The best known service is https://archive.org/services/purl/.

    Since 2016, the service is provided by the Internet Archive (blog post). From 1995 to 2016, it was provided by the OCLC.

    Lorcan Dempsey of OCLC welcomed the announcement as “a major step in the future sustainability and independence of this key part of the Web and linked data architectures. OCLC is proud to have introduced persistent URLs and purl.org in the early days of the Web and we have continued to host and support it for the last twenty years. We welcome the move of purl.org to the Internet Archive which will help them continue to archive and preserve the World’s knowledge as it evolves.”

    It uses several domain names, including purl.org, purl.net, and purl.com.

    You need an account on https://archive.org/ to create and manage your PURLs.

  • Another, younger service is https://w3id.org/, provided by a group of organizations that follow a social contract:

    There are a growing group of organizations that have pledged responsibility to ensure the operation of this website. These organizations are: […]. They are responsible for all administrative tasks associated with operating the service. The social contract between these organizations gives each of them full access to all information required to maintain and operate the website. The agreement is setup such that a number of these companies could fail, lose interest, or become unavailable for long periods of time without negatively affecting the operation of the site.

    They claim:

    All identifiers associated with this website are intended to be around for as long as the Web is around. This means decades, if not centuries.

    It uses the domain name w3id.org.

    To create and manage your PURLs, you need to submit a pull request on GitHub or send an email to their mailing list.

  • Some more.

Risk assessment

For the objective to get a permanent HTTP URL (with the ability to change the redirect target) without having to pay something, a PURL service would be the best choice:

  • Providing permanent HTTP URLs is the primary goal of these services, and their only reason of existence. Their whole focus will be on keeping these URLs working.

  • Providing such a service is not complex, and not hard on the servers, so there is a good chance that it can be kept online in the future, even with a very limited budget.

Other web services might also care about permanent URLs, but they have to care about much more stuff in addition, so their priorities are different, and they might have to discontinue their service because of commercial reasons.
As an example, take Google and look at how many services they discontinued (among them also services that provided URLs for their users’ content). And if there are businesses that could afford (and want) to keep URLs from unprofitable services alive, Google would certainly be among them, right?

Improve this answer
6
  • This is a useful suggestion, but note that OP is looking also for somewhere to host the data. A reasonably permanent URL for accessing the data is nice, but you'll still need some place to point it, which the OP seems to want to be free of charge.
    – user
    Commented Dec 26, 2017 at 21:00
  • @MichaelKjörling: Yeah, this post answers only the question in the title and the bold part. As OP says "The stability of the actual location is not as important as the stability of the link itself", I don’t think it makes sense to recommend a hoster here, as any gratis hosting service would do the job, given that the permalink can be updated.
    – unor
    Commented Dec 26, 2017 at 21:08
  • 1
    This makes more sense than a github repo
    – icc97
    Commented Dec 27, 2017 at 7:00
  • 2
    I think this neatly combines with @FranckDernoncourt's answer
    – icc97
    Commented Dec 27, 2017 at 7:04
  • isn't this what doi.org does too? With the added benefit that url = site + doi ?
    – jiggunjer
    Commented Dec 28, 2017 at 4:39
3

One recently launched service that addresses your problem is the Wolfram Data Repository:

The Wolfram Data Repository is a public resource that hosts an expanding collection of computable datasets, curated and structured to be suitable for immediate use in computation, visualization, analysis and more.

In the launch announcement, Stephen Wolfram writes:

With the Wolfram Data Repository (and Wolfram Notebooks) there’s finally a great way to do true data-backed publishing—and to ensure that data can be made available in an immediately useful and computable way.

In another part of the post, he writes:

Each entry in the Wolfram Data Repository has an associated webpage, which describes the data it contains [...] every entry also has a unique readable registered name, that’s used both for the URL of its webpage, and for the specification of the ResourceObject that represents the entry.

Regarding the size of the data sets, he writes:

There’s no limit in principle on the size of the data that can be stored in the Wolfram Data Repository. But for now, the “plumbing” is optimized for data that’s at most about a few gigabytes in size—and indeed the existing examples in the Wolfram Data Repository make it clear that an awful lot of useful data never even gets bigger than a few megabytes in size.

The announcement is very long and has much more about the rationale and vision behind this service and details of how it works. I couldn't find information about pricing -- presumably it's free for now -- or what promises Wolfram is making regarding the permanence of the data storage (except for the vague sentence "The Wolfram Data Repository, though, is intended to be something much more permanent"). But the service is fairly new so I expect those things will be clarified eventually. Wolfram Research is a serious company with high credibility in the scientific community and has been around since 1987, so this looks like an intriguing option for your data storage problem.

Improve this answer
1
  • 1
    Nice addition. I dont knew. But, they expect the data to be in the user account before submission, and a account that supports this size of data appears to be $103/month. wolfram.com/development-platform/pricing The vendor locked in is another point to consider, but out of the scope of OP question.
    – Cochise
    Commented Dec 25, 2017 at 17:35
1

I need to link to data used for a publication...I need at least a stable link to some place where the data can be located

Provide a link to your personal site and redirect from there.

E.P. raised the issue

Google Drive data is mutable - it could be altered by the owner at any point (and, conversely, viewers do not have any guarantee that the data they see five years after publication, if it is still there, has not been altered in the meantime). This makes it completely unsuitable for this purpose.

This issue is orthogonal to the OP's question, but nonetheless interesting. It can be solved by taking a cryptographic hash of the data and including that hash in the publication.

Improve this answer
2
  • As stated in the OP, I cannot and will not be personally responsible for the data.
    – errantlinguist
    Commented Dec 24, 2017 at 18:01
  • @errantlinguist, what does that mean?
    – user2768
    Commented Dec 26, 2017 at 9:15
1

DataPort is an initiative from the IEEE. You can host up to 2 TB and you will receive a DOI.

Improve this answer
2
  • 6
    It costs 2k USD to host a dataset that is open to any user of the service, while a free upload is only accessible to paid subscribers. That would not solve OPs problem. Also the site seems to be still in beta.
    – mts
    Commented Dec 24, 2017 at 12:12
  • They (temporarily, as of today 2019-11-11) offer the OPEN ACCESS DATASET for free.
    – Kamal
    Commented Nov 11, 2019 at 14:49
0

Nothing lasts forever, but free file hosting services exist even without restrictions on size. Nothing in the world is really free, so, these services would impose some other kind of restriction, e.g., advertisement, or noticeable downtimes, or low bandwidth, or discomfort for uploading or downloading, or really ugly and long (but stable!) URLs, etc. These services might also ask you for all your private data and sell it later or send you lots of targeted spam. Choose a service that causes you as little discomfort as possible. That would be my solution.

How to find such a service would be a different question. I usually first find a site comparing dozens of free hosting services and then take it from there.

Improve this answer
1
  • Whoever downvoted: why?
    – Leon Meier
    Commented Dec 26, 2017 at 18:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .