Assume you are knee-deep in a project, going through a series of papers that are seemingly relevant to your project. You found this ~10 year old paper, which looks very promising indeed. A lot of interesting results, good discussion, etc.

At this stage, I think it's obvious that you should check whether or not the results and conclusions in the paper are still valid by searching for any correspondence related to the article (e.g., pointing out that some claims do not hold, etc). However, I have yet to find anyone that digs into such information. You normally find a paper, read it, find more interesting papers through references and keep on digging in this manner until you have accumulated "enough" articles to form an educated opinion.

My questions are as follows:

  • Is this (or any other) type of quality assurance process, when it comes to cited literature, common practice?

  • Is there a way to streamline this process? PubMed does not seem to include all correspondence related to each article.

I should perhaps note that, if you are in biomedical sciences (especially related to complex diseases), about half of the publications turn out to be wrong or misleading after some years, and perhaps a quarter more are shown to be incomplete in its findings. I believe it's in the nature of biology, in contrast to more human defined sciences like mathematics or computer science. There are no formulas or calculations to check the integrity of the work published.


1 Answer 1


I think what you're looking for are the papers that cite a given paper. This is often listed as "Cited by" in various databases. If paper B shows that paper A is incorrect, paper B will almost always cite paper A. In that vein, you can look at all the works that cite a given paper to see if any disprove the results.

More typically, though, if a paper published 10 years ago is still relevant today, there are other more recent papers on the same topic which will either cite that paper or derivative works thereof, and you would use those more recent papers to ascertain the current state of the research. Old papers, particularly in quick-moving fields, tend to either fade away pretty quickly or serve as foundations for other research. If no one else has cited it in the interim period, it's probably worth taking a very critical look at why that is the case before throwing in all your chips on that single paper.

  • There are cases where something is seriously wrong/arguable with a paper (e.g. sciencedirect.com/science/article/pii/S0278691512005637) In such cases the correspondences/replies come in as separate entities, not just papers that cite and criticize the original paper. I have to disagree with you regarding articles becoming foundations due to continued validity, it could just as well be due to the fact that people care more for having references than for doing a QA on cited literature, or am I completely off the mark?
    – posdef
    Commented Feb 1, 2013 at 14:28
  • @posdef - Not sure what that linked article is, but there are links directly on that page to the resultant comments. Yes, they aren't peer-reviewed articles, but they are still academic publications, and they cite the relevant article in such a way that you could find it using a "cited by" search.
    – eykanal
    Commented Feb 1, 2013 at 14:42
  • 1
    @posdef - Regarding your second comment, I disagree; researchers usually don't just cite to get citationCount++. They will cite when the cited paper is relevant to their work for one of many reasons; examining a related aspect of the same problem, found similar/different results, provided a springboard for this problem, etc.
    – eykanal
    Commented Feb 1, 2013 at 14:44
  • 4
    Sadly, some citations exist solely as a defense against referees who might react badly to their vaguely-related paper not being cited, not because they're actually relevant.
    – JeffE
    Commented Feb 2, 2013 at 5:12
  • @JeffE true, besides I personally know PhD students that hunt down articles that seem to have relevant information in order to strengthen the interpretation of their results, without actually dissecting the work described in the paper in question.
    – posdef
    Commented May 21, 2013 at 11:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .