8

There are of course, a plethora of citation metrics. Are there any that consider how heavily cited a paper is within the papers that cite it?

Let's consider paper X.

One paper may casually cite paper X along with a bunch of other citations and never interact with it again.

e.g. "Much work has been done in this area of research (A, B, C, D, E, F, X)."

On the other hand, another paper may depend heavily on paper X: adapting paper X's methodologies for its own, utilizing the theory proposed by paper X, and directly comparing its own results to those of paper X.

To my understanding, a typical citation metric would pick both of these papers up as equivalent citations. Thus, paper X has 2 citations.

But that method misses so much of what paper X has contributed towards the 2nd paper.

Are there any metrics that consider this? Perhaps counting the number of times paper X is cited within the papers that cite it?

  • 7
    This is a cool concept. Every time I get cited I go and manually check how much they actually cite me. If you build a tool that does this, let us know! :) – Austin Henley Feb 5 '16 at 21:16
  • 4
    My brother had a similar idea: janlo.de/wp/2010/04/04/scientific-citation-markup – Dirk Feb 6 '16 at 11:39
  • I think I would also measure how far the next citation is located from the cited paper in the text. In your example each of (A, B, C, D, E, F, X) would get very little score. But if there's a sentence that says "We use the methods of X" then here X would get more score. – domotorp Dec 2 '16 at 9:59
3

As far as I know, no. The problem is that what you want to measure is impossible to accurately automate (even standard citation counts that Google Scholar automates are not 100% accurate) without hard-coding such information into the paper, as in Dirk's comment. This of course has some issues, in addition to being a paradigm shift for the way citations are made (just as there are pressures now to include a citation, there would be pressures with such a system to make something an "important" citation).

The suggestion you made, about counting the number of times a paper is cited, crossed my mind in the past (along with some other ideas) but the problem is that it doesn't directly measure what you're asking. There are different reasons to cite a paper, and the number of times you explicitly cite something in a given work depends on too many different factors, a large one being the author's writing style.

Remember the current paradigm: references are just that, references, which put the work in context for the reader and supply details that one uses. Their purpose is not to provide a measure of the value of paper or researcher. Once you go down the path of twisting things to measure worth and assign prestige (even if that is not your purpose), you just make things into a competitive game.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.