9

I am pursuing a specific research question. I have thoroughly surveyed existing research on the topic, and found dozens of researchers working on the problem, but was disappointed by their work. Their research:

  • Does not consider the full magnitude of the problem.
  • Constantly tests old, insufficient methods.
  • Overlooks significant details, so the test results are meaningless.
  • Lacks innovation.

I found lots of interesting ideas posted around the Internet. In blogs, forums, and USENET, I found people with some clever new ideas to approach the problem. These people had a genuine stake in the problem, so I found their ideas actually brought the problem somewhere meaningful. These informally-posted ideas need testing and considerable refinement. They are far from perfect, but many times better than what the academics are dealing with.

I would like to prepare some trials and publish some papers, centered around a number of these ideas. It is only fair that I give credit to the authors of those ideas. Essentially, I need to give credit to lots of anonymous people who posted their ideas informally. I have never read an academic paper containing highly informal references. Can I include references like this in my paper?

MutantTurtle17. “My Amazing DIY Tin-can Refugee Shelter.” MyBlog. 2014.
    Retrieved from http://...

SimCityFan2012. “RE: RE: Look at this!” Shelter Designs Forum. 2013.
    Retrieved from http://...
7
  • Is your question whether you should cite ideas from these sources or whether you should use them? Because certainly if you use them, you are required to cite them.
    – ff524
    Commented Aug 12, 2014 at 7:55
  • 1
    @ff524 How is your definition of "use"? If Village reads about a cool new approach to tackle a problem in a blog, it will influence how (s)he approaches the problem next, even if (s)he does not use the exact same approach. However, one can hardly cite everything one has ever read about a topic.
    – xLeitix
    Commented Aug 12, 2014 at 8:41
  • @xLeitix I mean "use in a way that typically requires citation," which does not depend on the kind of source it is (academic, informal, whatever)
    – ff524
    Commented Aug 12, 2014 at 11:06
  • One option that I have used before, is adding a footnote saying something along the lines of "Inspired by ideas discussed on thisblog.com"
    – dsfgsho
    Commented Aug 12, 2014 at 11:20
  • 2
    @ff524 But it does. You should not excessively "use" soft sources in a way that requires citation. If you do, you can easily run into the trouble that says that your entire research is based on shaky assumptions.
    – xLeitix
    Commented Aug 12, 2014 at 11:33

6 Answers 6

16

Does your paper really test & verify lots of ideas?

You state that "I would like to prepare some trials and publish some papers, centered around a number of these ideas." How many of those ideas do you expect to actually implement per a single paper?

If you implement, evaluate and contrast three novel ideas from blogs&forums, preferably including a solid comparison against a baseline published method; then that's just three informal items that you need to cite, in addition to the current academic publications.

There are references and references

If your paper assumes something, or claims non-obvious things, then you need 'proof' of it outside of your paper that should come from references. Those references need to be trustworthy - preferably respectable peer reviewed publications.

However, if your paper uses references for giving credit to ideas or pointing to original sources, then that's an entirely different class of reference, where blogs and forums are just as acceptable as, say, referencing archives of private informal letters that are used in studies of literature or history.

If you have never read an academic paper with a lot of informal references, then it is because it is very dependent on the field you're studying - for example, a thesis about racial stereotypes in online media would reference many informal sources as examples; while a thesis about particle physics wouldn't have any.

1
  • Why not then just split your references under two headers: References and Sources
    – gaborous
    Commented Aug 13, 2014 at 12:33
9

This is an actual problem that I also struggle with in some aspects of my research. There are problems in which the blogging and industrial world is, sadly, miles ahead of the scientific state of the art. However, citing an abundance of blogs and other non-reviewed resources is rarely a good idea. A few citations of web resources are usually ok, though. Hence, my (imperfect) solution to the problem currently is to cite the 2, 3 web resources that are best suited for my paper, and try to find academic resources that cover the rest of the ground as good as possible.

That being said, this situation is certainly a possibility for you. If you can take the ideas from these forums and blogs, and bring them on a sound scientific basis (e.g., through user studies or formal analysis, whatever is appropriate for your research) and publish it both scientifically and informally (e.g., in your own blog), there is a good chance that you make a strong impact on both the scientific side and the blogging community. At the end of the day, people tend to remember not only who originally threw a revolutionary idea or concept out there, but also (sometimes even more so) the person that made the revolutionary idea work (or, at least, clearly showed that it works).

5
  • 1
    Another point is that anything that is not peer reviewed cannot be considered "true", and thus, cannot be used to back up claims (except some very obvious and objective cases). On the other hand, how good is the place where you got your idea, if you did your job well, doesn't matter.
    – Davidmh
    Commented Aug 12, 2014 at 11:11
  • @Davidmh: I've repeatedly run into such situations, and the claim I was trying to back up was invariably something along the lines of "technique X is used in real-world applications of domain Y". For lack of any peer-reviewed surveys on all kinds of new product versions and their single features, citing the software package itself (or its official website) is often the only choice. At the same time, the statement that feature X exists in real-world software is proven to be "true" by the very existence of said software, despite not being peer-reviewed. Commented Aug 12, 2014 at 12:36
  • @O.R.Mapper that follows in the category of "obvious and objective". In any case, the fact that other people are using it is not relevant for the validity of the research (but it is always good to show that your work has down to earth applications).
    – Davidmh
    Commented Aug 12, 2014 at 12:49
  • @Davidmh: Maybe I am in a field that is rather close to real-world applications, but I have made the experience that showing that certain related work is used in real life rather than just defined in papers can mean the difference between acceptance and rejection of a paper. I think that is because of the very true statement that xLeitix made, "There are problems in which the blogging and industrial world is, sadly, miles ahead of the scientific state of the art.", so comparing new contributions only to other research is deemed insufficient by the respective reviewers. Commented Aug 12, 2014 at 13:09
  • It is absurd to assert that anything that is not peer reviewed cannot be considered true (or "true"). There are many many factual assertions you could make in research where the relevant reference confirming the assertion is a non-peer reviewed source (e.g., newspaper reports, legislation, court judgments, etc.).
    – Ben
    Commented Feb 8, 2023 at 21:45
3

Maybe one way of dealing with having lots of informal references would be to divide the bibliography into sections, so that the reader can easily see the different types of reference (acknowledgement versus justification).

Or alternatively maybe put them all in an extended acknowledgements section (since databases won't be able to do much with blog citations anyway, perhaps it wouldn't matter so much if they don't appear in the official bibliography, provided the reader is sufficiently informed of who came up with what).

3
  • An acknowledgment isn't the same thing as a citation so that doesn't seem to be a good solution. Commented Aug 12, 2014 at 16:15
  • 1
    I guess it depends on how you phrase it in your text. If you are saying "x is true because Y said so" then you are claiming that Y is a reliable source who would or should know if x is true or not. If you say "x could be true (source: Y) but let's test it to see" those are very different things.
    – earthling
    Commented Aug 13, 2014 at 13:50
  • It may be I've slightly misunderstood the question. I read it not as 'these people have essentially published research in an informal setting' (in which case I'd see this as a citation), but rather as 'these people have generated ideas for research that could be done/suggestions for possible solutions to a problem, but haven't done the actual research of testing them' (in which case I'd see this as acknowledging their ideas, whether as a formal citation or by stating this more explicitly).
    – Jessica B
    Commented Aug 14, 2014 at 8:06
1

If the post knowingly belongs to some well known researcher or otherwise a known, notable person, such post can be cited, because even "personal communication" at the end can be a reference. However it is not good as a proof that something questionable is true as this is not a peer reviewed article.

If the author of the post is anonymous or not a scientist, such source is not trustworthy and is only suitable as a raw input data for analysis in social research.

1

I am a mathematician, former editor of several journals (currently, just one). Suppose I were to receive a paper submission with an accompanying email saying something along the lines:

I have thoroughly surveyed existing research on the topic, and found dozens of researchers working on the problem, but was disappointed by their work. Their research:

Does not consider the full magnitude of the problem. Constantly tests old, insufficient methods. Overlooks significant details, so the test results are meaningless. Lacks innovation.

I found lots of interesting ideas posted around the Internet. In blogs, forums, and USENET, I found people with some clever new ideas to approach the problem. These people had a genuine stake in the problem, so I found their ideas actually brought the problem somewhere meaningful. These informally-posted ideas need testing and considerable refinement. They are far from perfect, but many times better than what the academics are dealing with.

Or/and, checking the bibliography list, I see many references to Reddit, Wikipedia, Quora, blogs by people I never heard of...

My crank-meter would go to something like 99% and my first reaction would be:

Should I even bother soliciting a quick opinion (let alone a referee report) on this paper? Or should I check if the supposed solution of the 4-dimensional smooth Poincare Conjecture uses "simply-connected" instead of "homotopy-equivalent to the 4-sphere?"

2
  • For many journals you don't need to submit a cover letter along the manuscript. Moreover the above can be said more subtly. Commented Feb 8, 2023 at 22:39
  • @MassimoOrtolano: True, that's why I put a connector "or/and". Yes, it can be put more subtly... Commented Feb 8, 2023 at 22:49
0

This is just a personal opinion, but let's run it up the flagpole and see who salutes.

When deciding whether to use (and therefore cite) information from a particular source, ask yourself: who has checked that the evidence and arguments presented in this source really support the conclusions this source reaches?

For a peer-reviewed paper in a journal or conference proceedings, the answer is "a couple of independent experts in the relevant field, in a formal procedure in which they're fully focused on the task". That's great, you can go ahead and use (and therefore cite) the information with only a pretty cursory check of plausibility on your own part.

For a Wikipedia article with many independent authors and a lively talk page, the answer is "a large community of Wikipedia users, some of whom are experts in the relevant field and some are not, who may not be fully focused on the task in a formal process, but who at least have a clear pathway to correcting any errors they discover". Again, this is pretty good - maybe you need to be slightly more careful in your plausibility check than you would with a peer-reviewed paper, but you're still good to use and cite.

For a book that's been through many editions, the answer is "a large community of readers, some of whom are experts in the relevant field and some are not, but who don't have any particularly clear or reliable pathway to correcting any errors they discover". Before you use (and therefore cite) this, you're going to have to do a bit of work checking the evidence and arguments really support the conclusions yourself.

For a book in its first edition, or for a newspaper or magazine article, the answer may be "a single editor who was more concerned with style and grammar and spelling than with the substantive validity of the arguments". Before you use (and therefore cite) this, you're going to have to quite a lot of the work of checking the evidence and arguments really support the conclusions yourself.

For a Wikipedia article with a single author and a moribund talk page, or for a blog or forum post, or for an ordinary web-page, the answer may be "no-one". You can still use (and therefore cite) it, but you're going to have to do all the work of checking the evidence and arguments really support the conclusions yourself, and put enough details in your manuscript/assignment to convince the referees/examiners who are evaluating your manuscript/assignment that you've done that work.

(In all cases, part of that work of checking might be done by investigating whether multiple independent sources reach the same conclusion.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .