# What is a fair metric for assessing the citation impact of journals across disciplines?

To quote Thomsons "a journal's Impact factor is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years."

I assume that disciplines vary in

• average number of citations per paper: Disciplines with fewer citations per paper will appear to have less impact.
• citation half-life: Longer half-lives means under-estimation of impact relative to journals with shorter half-lives. The Wikipedia article on impact factors summarises a study that found that "the percentage of total citations occurring in the first two years after publication varies highly among disciplines from 1-3 percent in the mathematical and physical sciences to 5-8 percent in the biological sciences." (Nierop, 2009).

Google Scholar uses the five year h-index. See this listing of top ranked journals with various psychology related keywords in their title. The five year h-index indicates the number of papers with an equivalent number of citations. E.g., a value of 20 indicates that 20 articles published in the last five years have received 20 or more citations.

However, while the h-index might reduce the issue of different citation half-lives, it does not resolve it. And it does not address the issue of differential citation patterns across disciplines.

### Question

What index provides both a reliable and unbiased assessment of the citation based impact of a journal when comparisons are being performed across disciplines?

### Reference

• Erjen van Nierop (2009). "Why do statistics journals have low impact factors?". Statistica Neerlandica 63 (1): 52–62. doi:10.1111/j.1467-9574.2008.00408.x.
• Defining field boundaries is an extremely difficult problem. There is some cool work on this via statistical properties of random walks that makes nice definitions, here. I recommend taking a look around that site. A naive metric might be: classify journals into their fields and then assign each a ranking p, meaning the journal is in top p% of its field by some standard metric (say eigenfactor, since I link to that site). Note that the website claims eigenfactor and article influence scores adjust for different fields, already. Apr 5, 2012 at 7:20
• @ArtemKaznatcheev That sounds reasonable, and I imagine that discipline specific rankings is a common approach. My first thought was simply to think of a metric that is less influenced by citation half-life and discipline specific citation patterns. Apr 5, 2012 at 7:34
• The reason why I asked it here is because this is a real issue in some psychology departments where research output is being compared with academic output from other fields such as the biomedical sciences. But I'd be happy for the question to be migrated to academia.SE if it was preferred. Apr 5, 2012 at 7:50
• I raised the question of migration in chat. Apr 5, 2012 at 8:02

There is no such index.

Publication and citation standards vary significantly between disciplines and even sub-disciplines. Without direct, deep knowledge of the standards in each community, it is simply impossible to compare impact of a journal in field X with the impact of a journal in field Y. (Eigenfactor's extraordinary claims to the contrary require extraordinary evidence, which they don't provide.)

Moreover, it's not clear why you should even try. Any judgement about the relative importance of Journal of X versus Journal of Y necessarily requires a prior judgement about the relative importance of field X versus field Y. HC SVNT DRACONES.

• And Thompson's is evil, to boot :) Apr 5, 2012 at 20:35
• More importantly, at least within computer science, they're simply incompetent. Apr 6, 2012 at 7:14

The CWTS SNIP factor controls for differing disciplinary citation rates and speed. See http://www.journalindicators.com It is also shown in Scopus

Here's an informal and difficult to quantify one: does it suffice to have a publication in this journal to get tenured? Or, in a reverse manner, can one get tenured without having published in a journal like this?

• In economics, you will get tenure in most reasonable places for publications in Quarterly Journal of Economics, American Economic Review, or Econometrica.
• In statistics, these would be Annals of Statistics, Annals of Applied Statistics, Journal of the Royal Statistical Society Series B Methodology, the Journal of the American Statistical Association, and Biometrika.

These are the fields I am closely familiar with. I can imagine that in most natural sciences, a publication in Nature or Science would boost your chances quite a bit. Finance has a system of ranking journals, with a handful of highly coveted journals designated as A-journals.

Getting reliable statistics along these lines is of course impossible. If Google were a little smarter, it would scan the CVs of those tenured and untenured (those who moved to another university after the typical period of 5-6-7 years), grab their publication records, and see what the journals are that those who got tenured had published in that untenured hadn't.