How to find free data sets from the internet for independent research?

Question

I am conducting an independent research on a tourism-related topic. However, data sets on the internet require premium registration.

I was wondering: what are the best websites that offer data sets for free? Can experienced researchers give any good leads?

Answer 1

First - looking for already published research projects that contain and/or utilize datasets similar to what your project requires might be a good start. There are some researchers and research groups that wouldn't have any issues sharing their dataset with you if you provide them with a good enough reason as to why you need it and what you're going to use it for.

That being said, because of privacy and data protection laws; it greatly depends on how they acquired the data in the first place and whether or not there is personally identifiable information within the dataset. If you don't do this right, you can get into quite a lot of trouble.

There are also a number of sites that supply anything from large national and international gov datasets to the more modest limited access sites.

Most U.S. government agencies provide open access to a large portion of their data.

The FBI's web services Data.gov The CDC The US Census Bureau ... the list goes on.

Kaggle is often talked about BuzzFeed, surprisingly, provides a vast array of open information data.world ... to name a few.

You may also wish to contact universities and college departments. The often work with massive datasets and are quite liberal when it comes to sharing.

Good luck :)

Answer 2

It's impossible to give a definitive answer without knowing your dataset needs. Do the datasets also need to be about a specific tourism-related topic? Are you assessing the effectiveness of a particular model, and must the datasets you seek accommodate that type of modeling? Perhaps we can give more specific guidance if we know more.

Generic sources include:

• the UCI machine learning dataset repository, at http://archive.ics.uci.edu/ml/datasets.php , provides 497 datasets for free
• if you're looking into a multi-label classification task, MULAN lists 26 datasets at http://mulan.sourceforge.net/datasets-mlc.html
• if you're interesed in multi-target regression instead, MULAN has 18 datasets here http://mulan.sourceforge.net/datasets-mtr.html
• the Journal of Applied Econometrics has a data publishing policy, such that datasets for almost all papers published in this journal since January 1994 can be accessed here http://qed.econ.queensu.ca/jae/

Hope this helps.

Answer 3

General Search Engines for Datasets

• Google Dataset Search, or
• DataCite Search.

(Governmental) Open Data Portals

Many governments around the world have created "open data portals" which allow free access to machine-readable datasets generated by the public sector (e.g. by national statistical institutes). Check out, for instance

• the European Data Portal,
• Open Data Portal Watch, or
• Open Data Inception.

Data Repositories

For non-public datasets, you can always search for open "data repositories" such as Harvard Dataverse. You can find directories at:

• Re3Data (Registry of Research Data Repositories),
• FAIRSharing, and
• Nature's Recommended Data Repositories.