7

I have seen several different metrics that are applied for ranking journals: Impact Factor, SJR, and H-index. Which of these is the most useful and robust for comparisons with respect to the quality and prestige of journals?

  • 3
    This question is not answerable as such. What does "reliable" mean to you? What is SJR? And I am not even sure whether the h-index is actually used to measure journal quality. – xLeitix Dec 31 '14 at 15:51
  • 1
    h-index is normally an author metric, but I could also see how you could make it into a journal article (e.g., a journal has an h-index of 50 if it publishes 50 papers in the last x years, each of which has acquired 50 citations). – aeismail Dec 31 '14 at 16:15
  • What I meant by reliable is the most robust, comparative and indicative of the quality, prestige of the journal. Thank you for your comments. – Ahed Dec 31 '14 at 16:47
  • @Ahed You should add these clarifications to the question, and possibly expand on why you are asking (evaluation of publication profiles, choosing journal to publish in, etc?). Readers of the question shouldn't have to go through all the comments to understand your question. – fileunderwater Jan 1 '15 at 23:12
  • I have clarified the question to match answers (addressing the "unclear what you're asking" closure reason), and nominated to reopen. – jakebeal Feb 15 '15 at 4:31
11

In short:

  • Impact factor is the traditional measure of mean citations per year shortly after publication. It can refer to either the official Thomson-Reuters computation (which has lots of problems), or other competing similar computations.
  • SJR attempts to improve on impact factor by using an algorithm like PageRank to more heavily weight citations from "better" publication venues.
  • H-index for journals is just the same as for people, essentially looking for the volume of consistently good work. The only large place I know of that applies this to journals is Google Scholar metrics.

All of these are subject to manipulation and gaming, all are highly field-dependent, and none are very good actual ratings of quality. Still, they can be used to get a rough sense of the significance of a venue that you are unfamiliar with.

  • Thank you for your brief, useful answer. I've benefited from this answer. – Ahed Jan 1 '15 at 20:31
  • 1
    Can you expand on the 'lots of problems' in Thomson Reuters IF? – Cape Code Jan 1 '15 at 22:56
  • 1
    @CapeCode Among other problems, it has atrocious coverage of computer science venues, the limited time window penalizes journals with slow review and publication processes, and disproportionately rewards "hot" work, even if it has no lasting impact. – jakebeal Jan 1 '15 at 23:01
  • 2
    @CapeCode Also, the IF figures that ThompsonReuters reports are not consistent with their reported algorithm run on their reported citation data. – JeffE Jan 2 '15 at 1:56
5

One difference between the traditional Impact Factor (JIF) (the official Thomson-Reuters score) and the SJR score (SCImago Journal Rank) is that the former is based on the ISI Web of Science database and the latter on the Scopus database. Depending on the field you are in this can translate to differences in coverage, which can affect the relative rank of journals. The SJR score is also a weighted score, which takes journal prestige into account, and this is not the case for the Impact factor.

The H-index is strongly influenced by the number of papers published by a journal, since this translates into the number of "attempts" of a journal to obtain highly cited papers. This is the main reason why journal rankings based on the H-index usually deviates quite alot from ranking based on avarage-based metrics (such as the Impact factor and SJR). When using the H-index to rank journals a small top-journal within a field (based on article impact) can be surpassed by large journals. As an example, journals from the publisher "Annual reviews" generally come out much lower on H-index rankings compared to IF or SJR-rankings. Personally, I find the H-index a poor proxy for journal prestige, but it might be somewhat useful to determine overall influence on a field (as in number of relatively highly-cited papers). Overall, I think that the H-index is much better for evaluating individual scientists than to compare journals. Why Google Scholar chose it as the only score to present when comparing journals is beyond me.

However, there also exists many other indicies of journal performance, influence and prestige. Hocking (2013) provides a comparison between 11 different journal metrics, using ecology journals as a case study. This study show that while all indicies are positively correlated, they form 3 relatively distinct clusters. The traditional impact factor is closely related to SJR as well as to Article influence (AI) and Source Normalized Impact per Paper (SNIP), while the H-index and the Eigenfactor are part of other clusters. This means that journal rankings based on e.g. JIF, SJR, SNIP will be very similar, while rankings based on the H-index or Eigenfactor will emphasize other aspects of journal performance.

References

Hocking. 2013. Comparing the influence of ecology journals using citation-based indices: making sense of a multitude of metrics. Ideas in Ecology and Evolution 6(1).

0

Among widely-available indicators, SJR and the Article Influence Score (AIS) are best for characterizing journals. An important property of those two indicators is that they weigh citations with the relevance of the citing journal, whereas in computing the impact factor (IF) or the h-index all citations are considered equally, regardless of the relevance of the citing publication. While all these metrics are typically highly correlated, taking into consideration the relevance of the citation sources can matter especially when trying to assess journals that are not among the top ones. For example, in the case of a journal that would artificially inflate the IF through self-citations or by participating in a citation ring within a publisher, or in the case of a journal hosting publications of a relatively isolated field, the SJR and AIS would remain relatively low if citations would come from low-quality sources. Another advantage is that the typical values of SJR and AIS vary less among different fields than the values of the IF and h-index. There are established methods for field-normalizing the IF (see, e.g., the SNIP indicator), however is more difficult to field-normalize the h-index.

  • SJR and AI do not include a measure of "relevance", if you by this mean the relevance of specific citations to the paper or the scientific field. Both indicators include a concept of journal "prestige" in their weighting of citations, but this is simply a single measure for each journal and does not equate to "relevance" (whatever that is). – fileunderwater Oct 18 '16 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.