Are there academic reasons to only use journal articles and books when writing?

Question

Recently I was researching for a paper, but the results of my research got me reflecting on my habits:

Is is at all appropriate for me to use Google (partly also Google Scholar) for academic research, just because I have good experiences with it in my personal, non-professional environment? I noticed I feel slightly too comfortable in sourcing material via Google. As expected, the quality of the resulting materials is "bad", in the sense that the sources I happen to find are rarely journal articles and often chronically difficult to cite. Of course for many topics this is a non-issue, but for my current topic, a lot of government, international institution and NGO content was "on-topic" and needed, so that is how the situation came about in the first place.

While I definitely won't use Wikipedia or (online) newspapers, but I also came across e.g. OECD content: while the content itself is strong and academically valid, by now I sadly had to discard a significant part of it, also because I initially didn't (exclusively) use their "iLibrary". Specifically: The amount of time I had to spend trying to find e.g. the author of such non-journal content (and other source-data), trying to decide if I should invest my time into reading and citing the content vs. discarding it and that I have to enter bibliographic data by hand for such content make me tired.

Thus my thought is: Should I intentionally, sternly refrain/refuse to use (and of course cite) any materials that doesn't carry a DOI, ISBN or ISSN? There is still a lot of grey literature, manuscripts, working papers etc. and web content out there... My current frustration is so large that I seriously consider such a strict stance going forward, but I fear that I might miss out on crucial facts and that omitting such publication will affect the credibility and well-roundedness of my paper(s)?

In the age of Google I am very happy that academic standards are in place, compared to the lack of cite-ability etc. in documents published by NGO's, think tanks, policy institutes and some governments.

Answer 1

If you build your work upon other people's work, whether it be a blog post or postage stamp, then you should cite it, otherwise that's plagiarism. This is especially the case if the work is a primary source of the research (which can even be the case for blog posts).

That said, there are some guidelines you can follow to get the best version of the work to cite.

• Find the most recent or most authoritative/official version of the work. If the paper is an unpublished workshop paper, is there a corresponding conference or journal paper that was later published? Or is the work perhaps written up in a masters or PhD thesis?
• Find the primary source. Is the blog post (secondary source) a distillation of some other paper (primary source)? If so, go for that primary source. Is the Wikipedia article (tertiary source) based on some well-known book or article (primary source)? If the report from an NGO is original research (primary source) then you should cite it. If not, determine what it is based on an cite that.

Answer 2

From my own personal experience, I have cited non-traditional sources that go far beyond the realm of books and articles, and even repositories like arXiv.org.

In some cases, these included "trade association" documents that cited chemical compositions for blends that we needed to have in order to set up a numerical model. In that case, the document was "official" enough that it could pass muster as a viable source of the data. Similarly, if one wants to cite a source like the NIST Webbook or an online database like the Merck Index, those are clearly curated well enough that one doesn't necessarily have to worry about judging its validity.

For sites that are not so well curated, then you do have to work as an arbiter of the quality of the website. You'll need to verify if there is sufficient data and evidence to back up the claims that you find in the documentation you want to cite, and if the document is suitable for citation, or if you need to dig further back to find better "original" sources. So long as you can "trace back" your work with suitable confidence, it's probably OK to cite a source from the Web. Once the provenance or accuracy becomes nebulous or tenuous, you should look for alternate (or more primary) sources.

To go the other way is, of course, also possible. The challenge will be that if there are very important sources of literature published outside the journal system—for example, government reports or other standard materials that "define" their field, then you will probably be called out if you omit them—and their findings—from your work. Now this may not be true in your field, in which case this won't be a concern. If you get a referee report with such a request and refuse to make the change on the ground that it doesn't have a digital identifier, you may find yourself with a rejection notice at the end of the day.

Answer 3

DOI, ISBN and ISSN are very useful tool for managing citations, but nothing more. You need to cite every material you rely on. Otherwise it is strongly against scientific ethics (i.e. plagiarism or intentional lack of acknowledgement).

Of course, when you use materials without DOI, ISBN or ISSN you need to put special effort to properly identify the cited material and to have any reasonable chance, that it won't disappear.

But for example arXiv ids, or even links to MathOverflow answers are good candidates.

If you point is "I refrain form citing anything without DOI, ISBN or ISSN just because" then, well, someone can pledge not to cite your journal, or just - any of your works, because why not (and it happens on the same moral ground)?

Answer 4

Interesting question. I think the answer lies in looking for what purpose you cite "anything but journal articles and books".

In this respect, it is important to distinguish between primary and secondary sources in research reports:

• Primary sources are all items which were an object of your research, i.e., your research results directly stem from an analysis of these items. An example would be a text where you studied usage of a specific word. Or a telephone book, if you did a statistical analysis of telephone numbers. Primary references are rather rare in the natural sciences and engineering, but they would be more common in social sciences and humanities.
• Secondary sources are items from which you cite research results in order to support your arguments. For example theories that you base your hypotheses on, reports on the validity of methods that you are using in your research, or other research results which you discuss in relation to yours.

Obviously, for primary sources, it doesn't matter whether they have been produced with academic standards or not. After all, they are the objects of research, and the validity of your research results does (in principle) not depend on the validity of the primary sources.

On the other hand, the validity of secondary sources is crucial for the validity of your research results, or at least your interpretation of the results. That's why you only want to use secondary sources which are up to academic standards, i.e. peer-reviewed scientific publications and books from established authors / publishers.

So the conclusion is that you should in fact try to refrain from using less credible references as secondary sources, while it will be perfectly fine to use anything as a primary source.