Is it appropriate to cite vulgar websites used for gathering data?

Question

I am working on a master's thesis in natural language processing wherein I analyse various tokenisation algorithms. These algorithms keep a large fixed-size set of recognised words (more specifically, word pieces) in storage, on the order of tens to hundreds of thousands. Along the way of my analysis, I noticed in one of the non-English tokenisers I am studying that it had quite a few word pieces which could only be completed to form English pornographic terms (let's say ussy, inces, cking and so on). The reason would be that they were trained over web corpora that contain such webpages.

There exist lists of swearwords online that contain some of these words. However, there are also a bunch of words that qualify as pornographic genres (maid, POV...) that aren't inherently pornographic, were it not for the fact that they are English words appearing online in a corpus that largely consists of a different language and that has the obviously pornographic words already present in it. As further evidence, the names of some porn websites also occur in the tokeniser's vocabulary.

I find myself in a strange and slightly blush-making predicament, now: not afraid of the internet, the only place I could actually find a comprehensive list of pornographic genres and website names was, well, on websites that also host videos of those genres. Since this one experiment fits very nicely with the rest of the thesis and I find the results significant and interesting, I would like to feature it.

Proper academic methodology requires that my experiments be reproducible... but how do I cite my sources in the most appropriate way possible? I have only found one paper that speaks on this topic, which obfuscates their sourcing:

We filter out documents that have at least three types or ten tokens from a list of words highly used in pornography. The list was derived from the analysis of pornographic pages harvested in a previous crawl.

The lists of swearwords are all Node.js packages, so citing those is no problem. For the rest of the list, concretely, how does one cite porn.com in a scientific work, as appropriately as possible? Hyperlinking to it seems like a bad idea, and showing the URL to me seems "tasteless", although not showing the URL feels too handwavy and might raise even more eyebrows ("I use a list compiled from a variety of websites known to me").

---

This is at a Western-European university, so there won't be a moral inquisition if I include these results, but that doesn't mean it's not something to be nonchalant about. I want to respectful to the reader but academically precise.

Answer 1

Citation is not the same thing as describing your methods. It sounds like you need to describe your research methods, not use citations. Those research methods may include listing URLs that you accessed to gather data.

You will want to make sure that your methods are not against the terms of service of any of those sites you used. Just because you are able to view certain content does not mean you can do whatever you want with it; some uses may fall under the concept of "fair use" or similar depending on what legal jurisdictions apply to you.

I'm not familiar with your field, but I've certainly seen popular press that makes reference to well-known pornographic websites, and they do not bother to obscure anything about their names. These are big businesses, and you'll find them referenced in newspapers and news television.

I think there is a certain level of advice that I want to give along the lines of "grow up and get over it". If your research is going to involve pornography, you're going to have to get a lot more comfortable with your research involving pornography. Urologists, gynecologists, and obstetricians can't get all giggly every time someone refers to genitals, they work there.

If you have a reasonable research question that involves pornographic language and you've used pornographic websites in work towards those questions, you need to be sufficiently explicit about it to allow someone else to recapitulate your work. That's the standard in research. Long lists might make more sense in an appendix or similar data supplement, but if you got your list of naughty words from "porn.stackexchange.com", and you think your work is rigorous enough to be published, then mentioning that your words came from "porn.stackexchange.com" should be just fine. You can try searching Google Scholar even for the specific websites you have in mind to see how others have referred to them.

Answer 2

There is no room for bashfulness in research

As the other answers here point out: (a) if you require information for your research that can only come from pornographic websites, then you are going to have to source that information from pornographic websites; and (b) if you source information from pornographic websites then yes, you either need to cite those websites or at least record their use as relevant sources in the information in your methodology. In some cases where you are sourcing large numbers of individual webpages you might just cite the general sites themselves and have supplementary materials setting out all the specific links, etc., for the data sources (as opposed to listing them in the bibliography). In any case, the rule is that your reader should be able to track your data back to its original source to verify the accuracy of your work.

Beyond the academic requirement to show your sources, good data science practice is to save your data/sources locally if possible (within reason) and have a good system for tracking your data back to its original source. This might even mean that you would be well advised to save some pornographic pages or search outcomes in your local data folders to ensure that you can demonstrate the source of the raw data (in cases those pages change or become unavailable later).

Here I think you have actually identified an opportunity for some useful additional research that could form a separate paper. The predicament you describe seems to be one that many researchers examining language models would run into, and if you are correct that there is a sparsity of existing research on pornographic terms then you have a golden opportunity to be the one to do that research and write a useful paper with supplementary materials giving a list of terms that other researchers could use in the same situation you are facing. If you were to do this then you would need to wade through some pornographic pages for the initial project, but you would save later researchers the trouble of doing this. It would also mean that your existing language-modelling paper (and any other language-model papers in the future) could directly cite your other paper on the analysis of pornographic terms without having to cite any pornographic websites itself.

Answer 3

The basic rule in research is that if you use it, you cite it.

So, if you use data taken from a site, you should, indeed, cite that site.

I'll guess, however, that the basic structure of your research (vulgar words) will lead people to expect that you didn't find them in "proper" places. And, those places/people that would object to such a citation will object to your work in any case, unless you avoid using those words, even if obfuscated, in publications.

And, you are using such sites not for pornographic purpose, but for analysis of language. Those sites aren't secret from the public in any case.

Answer 4

Not claiming that this answer is canon or anything, but, if it were me, I would go for a solution that allows me to cite/link to the resources (for verification/reproducibility purposes), without necessarily including any of the vulgar content to the paper directly.

In other words, something along the lines of what you see in websites/tv shows when you try to enter and they warn you of potentially offensive content, e.g. something like this at the end of your bibliography:

[28-35] Please note: some of the resources used for this research unavoidably contain content that may offend readers, therefore citations to these resources are provided as a separate bibliography section. Readers wishing to access this for verification and reproducibility purposes can do so here: http://link.to.some.textfile.online

(where 28-35 refers to the citation number in the text, and where you make sure to lump all the 'risky' citations under those numbers at the end of your bibliography)

Furthermore, you could probably consult the journal editor on this issue; they may have a policy on this already, or advise you of their preferred solution (e.g. a particular repository to upload your bibliography section to).

Answer 5

Don't

If I was doing a computer analysis of language from novels from the 1850s I would not include the list of titles in the paper itself. I would include it under supplemental material.

Since you are doing a crawl, I presume you are using a large number of websites. If there is a large list of websites then it is appropriate to not name them in the paper itself, but to just describe the websites, and include the list of domains or whatever in the supplementary material.

Answer 6

Why not? I work in econometrics. Causal inference, specifically. There are two quasi-experimental papers I've wanted to do that directly or indirectly involve PornHub. So long as your work is academic, then this shouldn't matter if the question matters. Moreover, we're all adults. So, it's okay.