Are there any metrics considering how heavily cited a paper is within another paper?

Question

There are of course, a plethora of citation metrics. Are there any that consider how heavily cited a paper is within the papers that cite it?

Let's consider paper X.

One paper may casually cite paper X along with a bunch of other citations and never interact with it again.

e.g. "Much work has been done in this area of research (A, B, C, D, E, F, X)."

On the other hand, another paper may depend heavily on paper X: adapting paper X's methodologies for its own, utilizing the theory proposed by paper X, and directly comparing its own results to those of paper X.

To my understanding, a typical citation metric would pick both of these papers up as equivalent citations. Thus, paper X has 2 citations.

But that method misses so much of what paper X has contributed towards the 2nd paper.

Are there any metrics that consider this? Perhaps counting the number of times paper X is cited within the papers that cite it?

Answer 1

As far as I know, no. The problem is that what you want to measure is impossible to accurately automate (even standard citation counts that Google Scholar automates are not 100% accurate) without hard-coding such information into the paper, as in Dirk's comment. This of course has some issues, in addition to being a paradigm shift for the way citations are made (just as there are pressures now to include a citation, there would be pressures with such a system to make something an "important" citation).

The suggestion you made, about counting the number of times a paper is cited, crossed my mind in the past (along with some other ideas) but the problem is that it doesn't directly measure what you're asking. There are different reasons to cite a paper, and the number of times you explicitly cite something in a given work depends on too many different factors, a large one being the author's writing style.

Remember the current paradigm: references are just that, references, which put the work in context for the reader and supply details that one uses. Their purpose is not to provide a measure of the value of paper or researcher. Once you go down the path of twisting things to measure worth and assign prestige (even if that is not your purpose), you just make things into a competitive game.

Update (2021) (though it's been in use for a few years now) There is at least one algorithm that tries to classify important citations. See

https://www.semanticscholar.org/paper/Identifying-Meaningful-Citations-Valenzuela-Ha/1c7be3fc28296a97607d426f9168ad4836407e4b

It is used in Semantic Scholar (check it out with the above link) to count, e.g., "Highly influential citations." However, looking at citations to and from my papers, it isn't too accurate.