I'm doing a research project where I want to collect thousands of screenshots of different websites, for training/testing of image models. Ultimately, I am trying to create a dataset that I can present and distribute to the community at various conferences/symposia.

My primary concern is how legal this might be. My assumption would be that taking a screenshot of websites of private companies like newyorktimes.com would be some sort of copyright violation, but that may be a faulty assumption.

I'm also unclear as to grey areas like:

  • reddit: user provided content
  • twitter: user provided content
  • wikipedia: open source knowledge base
  • google: content scraped/sourced from other providers

Essentially, what copyright considerations, if any, would I have to keep in mind when collecting, annotating and distributing screenshots? How might licensing/attribution work when making this data publically available to others for use?

  • Would it not be equally effective (and legally safer) to share the code of your web crawler so that others can easily re-create the dataset?
    – Thomas
    Jul 7, 2018 at 0:31
  • It would be, but part of what I'm aiming to do is annotate these images as well. My proposed dataset would include the images with said annotations, so just releasing my crawler doesn't quite get at what I'm aiming for.
    – mr.plow
    Jul 9, 2018 at 15:26

1 Answer 1


Given the state of copyright law, this sounds like a minefield. There are people trying to sue Google, just for indexing sites.

It isn't so much that you want to do research on these "images", which is likely ok, but I think that distributing them will be almost impossible. Many sites won't want to give you permission and will complain (via lawyers) if you haven't obtained permission.

The general law about copyright is that people own what they produce, whether they register a copyright or not. If there isn't a clear license offered such as a Creative Common license, then the owner retains all rights to the work unless they explicitly put their creation into the public domain. Most of the sites you will probably want to visit likely will NOT have given up rights. The deal is supposed to be time limited, but that idea has been nearly eliminated in the law.

Places like wikipedia have clear statements of licensing and rights transfer. But individual users at reddit, twitter, etc. likely still have rights unless they have yielded them (as part of signing up) to the hosting site. I don't participate much in social media so haven't investigated these user agreements.

Attribution isn't enough. You need permission. Once you have that, you need to provide attribution.

However, if you want to "process" images, gleaning characteristics of the sites (but not text and images...) you can probably do it without permission.

Also note that copyright law is in flux and varies greatly around the world. The EU is currently trying to work out very restrictive laws, for example.

The core problem is that IP owners often have a lot of political power. Disney, for example will declare war on you if you try to compromise their rights to Mickey Mouse. Common understandings that once permitted Voltaire to make a living from his writings have now expanded to unlimited rights to lock up the culture forever by large commercial firms.

Another problem is that many formerly lucrative publishing strategies have been money losers lately because of the internet. The NYTimes, for example, if it wants to continue in business needs to achieve income somehow. They can't do it printing on paper anymore, so their web sites have value to them and they are fearful (maybe overly fearful) that re-publishing, or even indexing, their material will cost them enough that they can't sustain their business model anymore.

It is even worse for sites that show advertising. You not only have issues with the site itself, but all of the images shown by the advertisers are, themselves, likely copyright protected with teams of lawyers ready and willing to go after you.

That said, you might be able to do your research on a wide range of sites/images without formal permission, depending on what it is, but then only publish a smaller set for which it is clear that you have a right to do so.

Note that there are a few exceptions to strict copyright law. These fall into the realm of formal criticism, and parody. There is a notion of fair use that is being eroded, so can hardly be depended on anymore.

If you are part of an institution, such as a University, then the institution probably has an Office of Research that can give you explicit advice for your locale. That office probably has access to lawyers that can provide explicit legal advice. Tread cautiously.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .