Should we trust a journal article that does not cite any previous work?

Question

My area of interest is not so well-researched in the US, but it attracts more interest in some developing countries where the topic is more relevant, but also where academic standards tend to be different, and I must proceed with much more caution.

In EBSCO, recently my searches have turned up a number of articles published in journals abroad that either: (1) contain no in-text citations, but present entries in the reference list related to material discussed earlier in the paper or (2) contain no citations at all and no reference list at the end.

Is the lack of citations in an article sufficient criteria for casting aside a paper, even if I think some of the ideas relate to a paper I am writing?

Answer 1

I think it depends what you mean by "trust".

A modern academic paper which references no other paper is unusual and possibly fishy, but at least in my field (mathematics) there is nothing inherently faulty about the practice: perhaps you really are answering a question that you thought up yourself and for which the answer does not require you to use or cause you to refer to any published result. (There must be some examples of this: anyone?)

If you're being asked to evaluate a paper then, sure, read it much more carefully if there are no citations. That's not a good sign. If you know the field well enough to know specific papers that the paper you're reading should have cited, then you should evaluate the paper negatively for not citing them. If the paper has been published in a reputable journal you might consider writing to the editors to suggest missed citations.

On the other hand, a lack of references (or in-line citations) does not cause a paper to cease to exist. If you want to cite the paper then you still can, and if you do use the material in the paper then you must cite it, as usual.

Answer 2

If you strictly ignored papers that had no references, then one of the papers you'd be ignoring would be Einstein's 1905 paper in which he originated the theory of special relativity. But times have changed since 1905, and it is now extremely unusual to see any academic paper without references; high-quality journals simply wouldn't consider such a paper seriously. Because of this, my experience has been that when kooks write papers, they do include references. From the examples I've seen in my field (physics) the telltale signs of a kook paper are not a lack of references but one of the following.

• Most of the references are to the author's own previous work.

• Where other people's work is referenced, the references don't show familiarity with the current state of the field. Instead, the references are to papers from the 1940s, or to sources such as textbooks.

Answer 3

Is there any kind of formal or empirical validation of the information that the paper contains? Citations and peer-reviews are standard today, as well as the reputation of the venue. Trust is often correlated with these, but this leads to the perversion of the whole system.

Many systems pervert when there is some discrepancy between how things are and how they look. E.g. economic bubbles, perceived and real value of some goods.

IMHO, trustworthiness comes from verifiability (and I think that is the whole point of science). Verifiability implies replicability, reproducibility, falsifiability, etc.

In short: no, you should not trust a paper that doesn't cite any previous work, but you should not trust a paper that cites many others just because they are there. You should, in general, trust no one, and check whether what they say is true by yourself, whether it is coherent with latter studies, whether latter studies could have contradicted it, whether it is possible to see if what they claim is true or not (open data, open science, open source, etc.)

Related and recommended: Top Ten Reasons to Not Share Your Code (and why you should anyway)

PD: a whole different point (as pointed by @petelclark) is evaluating a paper, in that case you have to consider whether what the paper says is true, but you have to check as well whether it is original research (or it was published before). Citations help to understand the context and the state of the art previous to the paper, and to see that the authors know the state of the art.

Answer 4

There are certainly non-traditional publications that have a high value and that you may wish to cite. For example, one of the important driving scientific analyses in biotechnology right now is the Carlson Curve, which is the DNA synthesis equivalent of Moore's Law. The up-to-date versions of the analysis, however, are not published in any journal article that I am aware of, but rather on Rob Carlson's blog. Not only is it not a "proper" article, but it has no "proper" citations---just links to various sources and related work. As such, it clearly preserves the intent and the value, just not the form. It's also not even vaguely peer reviewed, but it's an important work that is sometimes necessary to cite.

For the cases that you describe, coming from areas where the academic traditions are weak or different, it will typically be harder to determine the difference between low quality work and work that is high quality but fails to conform to the norms of scientific presentation. You should not discard something just because it fails to follow forms.

You should, however, treat non-conforming publications with heightened suspicion: publishing according to typical standards in well-known peer-reviewed publications can be viewed as a form of costly signalling. Conforming is no guarantee of quality, and failure to conform does not prohibit it, but failure to conform with publishing norms is strongly correlated with poor quality work for many different reasons.