I am wondering if there have been any sort of rigorous/meaningful study about how well an h-index measures the scientific productivity of a person. I am wondering how much stock I should put into the h-index, both as a personal measure and a measure of those around me.
-
19Having the ability to quantify how well the h-index measures scientific productivity would by proxy imply that there exists some reliable measure of scientific productivity to use as a control. Unfortunately, no such reliable measure exists.– MoriartyCommented Mar 16, 2015 at 15:13
-
5Is the h-index intended to indicate productivity at all? I think a researcher can be tremendously productive (in terms of publication frequency, number of findings, etc.) and yet receive very few citations for one reason or another.– O. R. MapperCommented Mar 16, 2015 at 15:16
-
Relevant ( see the answers for why not to do this) : academia.stackexchange.com/questions/5687/…– seteropereCommented Mar 16, 2015 at 18:31
-
Why on earth would you want to measure scientific productivity anyway, as opposed to scientific impact, or better yet, scientific insight?– JeffECommented Mar 17, 2015 at 21:36
-
@JeffE, depending on how you define scientific productivity it could include those aspects as well. At least that's what I've been thinking, maybe I'm wrong and that's what's producing some of the confusion here.– NeutronStarCommented Mar 19, 2015 at 15:31
Add a comment
|
2 Answers
Besides the convincing argument in Moriarty's comment to the OP there is also a formal argument that the index is particularly meaningless. The Notices of the AMS recently published "Critique of Hirsch's Citation Index: A Combinatorial Fermi Problem" by Alexander Yong, see here http://www.ams.org/notices/201409/rnoti-p1040.pdf
Roughly: The h-index is about half the square root of the number of citations...
More precisely: For an author with N citations to his paper, the h-Index always lies between zero and the square root of N. Now assume that these citation are distributed on a number of papers and all partitions occur with equal probability. Then it turns out that the estimated h-Index is then about 0.54 times the square root of N. In the article it is quoted that Hirsch himself has observed that basically all h-Indices he looked at were in the range of 0.45 and 0.58 times the square root of N. Yong also states that, especially for the highly cited people in math, physics, computer science and statistics only very few people have an h-Index above 0.54 times square root of N (and they are never more that 5% above that). The article contains much more data but only for famous mathematicians such as Field's medalists or members of the National Academy of Science and it is really worth a read.
-
"Roughly: The h index is about the square root of the number of publications..." Is this math-specific? Because for CS that seems awfully low.– xLeitixCommented Mar 16, 2015 at 21:04
-
1Oops, should have been "number of citations". Corrected. But then this applies universal - check the paper, it is really worth a read!– DirkCommented Mar 16, 2015 at 21:18
-
Shouldn't it be that the h-index is roughly half of the square root of the total citations? @xLeitix One of the early and easy-to-understand discussions is here: michaelnielsen.org/blog/why-the-h-index-is-virtually-no-use It was empirically observed in physics (not math) by Hirsch himself when he proposed the index. The AMS article gives more insight into it through combinatorics and observes that (a more elaborated version of) the rule of thumb works rather well for pure math. I think it should work more or less alright for other fields if citation patters are similar. Commented Mar 17, 2015 at 4:01
-
1[ (citations)^(1/2) ] / 2 indeed seems a reasonable approximation for the h-index for most researchers I checked. For really large h-indices (> 40 or 50) the rule seems to get iffy, though.– xLeitixCommented Mar 17, 2015 at 7:03
-
1The AMS article used a simple random model for the citation distribution, and gave a simple one-variable estimate (i.e., about 0.54 of the square root of the total citation count). And it turns out It doesn't seem to be far off in pure math for the cases they checked. This estimate seems to work ok for physics as well, which might suggest that how physicists get cited isn't that different than in math. In any case, if there is a typical citation pattern in a field and if you model it well, it's no wonder someone's h-index in your field typically follows this kind of one-variable estimate. Commented Mar 17, 2015 at 8:15
I'm no expert on this, and this is just a comment that is a little too long to cram in the comment section.
Before addressing your question, h-index is not supposed to measure productivity alone. It attempts to measure both productivity and impact in terms of citation count. The simplest and crude measure of the former would be the number of papers a scientist has published, and one simple measure of the latter would be the total number of citations. You could say these are two crude measures of quality and quantity, although their meanings are vague. In a sense, h-index is intended to measure the quality and quantity of scientific output by a single number.
Anyway, to answer your question, assuming that you are asking if there's any serious research on how well h-index works as intended, if you just google with some obvious keywords, you should be able to find many serious academic papers that study h-index, e.g.,
or several papers cited in this article (pages 5-6)
or many other papers you should be able to find through usual means like checking the references at the end of a paper on the topic, looking up papers that cite them, putting more effort in googling, trying different keywords you couldn't come up with at first but you can after reading some papers on the topic, using different search engines, etc.
Note that the above parers are by no means meant to be representative examples. I just googled a bit, skimmed bits and pieces, and copy-pasted the links. As I said at the beginning of this post, I'm no expert and as clueless as you. For what it is worth, the journal Scientometrics seems to have a decent impact factor, since you seem to care about that sort of thing.
I hope real academics who work on bibliometrics chime in and give expert answers.
answered Mar 16, 2015 at 18:05
