Ethics of using public and illegally obtained databases for academic purposes

Question

I'm on my senior year of college, and my professor and I have decided on an independent study where I will analyze and develop some method(s) for screening valid data sets for the 'selection' phase of data mining, where spam bots, dead accounts, and other irrelevant data sources are omitted from machine learning analysis.

The problem i'm running into is that I really want to use a data set that is from the real world, so I could truly discover something meaningful from the data that hasn't been discovered before.

My initial thought was to use the Ashley Madison hack dump from the summer of 2015, but I wanted to consider the ethical implications of using such sensitive data. Alternatively, I was thinking I could manipulate the data before analyzing it to provide some sense of anonymity to the victims of the hack (for instance, replacing all full emails with the first and last character of the name, as well as the @ and suffix).

My question is NOT whether you think these practices are ethical, but whether there has been some professional work done in the past anyone is aware of that can serve as a model for my current work.

For example, Facebook was caught manipulating the content of its users' Facebook feeds in order to measure the emotional reaction of the content on their future posts.

Answer 1

Relating to just the AshleyMaddison data, there is a plethora of articles on your exact question. I will summarize the main talking points and provide links at the end.

1. Can I download it?

The data is probably public-domain, but it depends on where you live. In some countries like the US, the data itself is publicly accessible and part of a wider conversation about personal rights to privacy, whilst in others like Canada, is has been explicitly decided that AshleyMaddison still holds the copyright and downloading the data is akin to acquiring stolen property (note, AM is owned by AvidLifeMedia, a Canadian company, and people have said that this decision was to offer some degree of protection to ALM, as hacked users cannot use the data dump as evidence in court). Other countries say that fundamentally nothing that can be downloaded can be stolen, so this doesn't even apply.

2. Can I share portions of the data with others?

If you are in a jurisdiction that allows you to download it, you can by default share it - and many (perfectly legal) sites exist for just that. You type your name or e-mail address in to see if you are part of the hack (or your spouse was). For better or for worse, there is no crime for making public-domain data easier to access.

3. Can I process the data and share summary statistics?

Oddly, unlike the above 2 issues, this is legal in every country which offers protection for journalists and researchers - including Canada. Many researchers, particularly those researching infidelity, have asked lawyers this question, and they all say the same thing - yes it's legal. In fact, top US lawyers have gone as far as to say that journalists are probably in the clear if they publish a list of names of celebrities who appear in the hack, for whatever little public good/interest there might be there.

Many articles also point out that there is a big difference between the legality of this, and the ethics of this. Both the law of the land and what is considered ethical behaviour changes over time, and they don't always have to be in sync. Some say that in using the data you are condoning and even encouraging the hack - which may lead to more hacking/data dumps in the future. Other's say that your research may be the only grain of good to come out of the whole debacle.

I will summarize by saying what I would personally do, which is use the data, get the outcomes, then way up the pros and cons of those outcomes with the example you will be setting for others in using this data. Legality is really not the issue here at all, because even if it is illegal, you are incredibly unlikely to find yourself going to court from either Ashley Madison or hacked users, as both parties would have a very poor case. Those aren't my words, those are the words of Jennifer Granick, a law professor at the Stanford Center for Internet and Society.

So if the real question here is the ethics, then this is something that is ultimately up to you to decide on. There may be all manner of real repercussions from your department - particularly if someone in the department turns up in the database... - but that's a very different question.

http://www.huffingtonpost.com/entry/ashley-madison-hack-creates-ethical-conundrum-for-researchers_us_55e4ac43e4b0b7a96339dfe9

http://www.al.com/news/index.ssf/2015/08/is_it_illegal_to_download_the.html

https://onlinejournalismblog.com/2015/07/20/ashley-madison-ethics-journalism-hacked-documents/

http://fortune.com/2015/08/19/ashley-madison-media/

Finally, there are many stories that people have posted online detailing their experience at the hands of the hack. Some say it wasn't really them, some say it was. Some are outraged, some are just numb. I would suggest reading one or two of the longer blog posts to really get a sense of what this data means to some people. It's more than just a resource. People have committed suicide due to the shame or discrimination they faced as a result - most notably people from the LGBT community - so it's really important to not shy away from that when deciding to proceed, or not, with your research.

Answer 2

You have no legal right to use this data in your research. Basically what you are describing is gaining unauthorised access to sensitive personal data and using it without the permission of the data holder or the individuals. The fact that some hackers posted the data on the internet makes this unauthorized access really easy, but it doesn't change what you are doing from a legal perspective.

Anonymising the data in some way doesn't change this. There are certain cases where anonymising data makes it legally usable for certain purposes--but not in this case, where you have no right to use the data in any way.

Using this data would put you at risk of prosecution and probably jail time. And, if you plan to do some research and try to actually publish it, I'd say this is a very real risk. Even if you avoid legal consequences, it's quite likely this will affect your ability to publish the work.

Caveat: I'm not a lawyer. Ask one if you want more information.

Answer 3

The answer of @dan1111 covers very well the ethics/legality of using this particular data set. To answer the question: are there standard methods of handling the human subject aspect of the proposed research? (with respect to your instinct that you should anonymize the personally identifiable data in it). I will assume you have a similar public dataset that was legal and that you did get the permissions to use.

The answer is yes. But at least in US universities, what you propose is classed as human subject research because of the personally identifiable information. Your research plan would include strategies on how to protect the human subjects, (protecting the the personally identifiable data). And it would first be submitted to the university internal review board (IRB) before you touched anything. They might expect a plan that ensured that the PII be anonymized even before you get to 'view' the dataset. What you propose might be enough, but they may require more steps or protections in place.

There are several questions that are similar on stack exchange and the answers will also mostly jump to quickly point out the equivalent of IRB approval. Ethics of scraping “public” data sources to obtain email addresses, Sensible measures to ethically use freely available, but personal web-based comments in research?, Is it legal/ethical to use data grabbed from a Stack Exchange site in a paper?