33

During last 3 years I have published 17 articles, but I have only 15 citations (excluding self citation), which I find disappointing.

My articles have 100s of downloads almost every month. They seem to be in high quality journals: 2 articles are in Q1 journals (i.e., a journal that is among the top 25% based on avg JIF percentiles), 5 in Q2 journals (i.e., 25-50th percentile), and the rest in Q3 journals (i.e., 50-75th percentile). I share my work on LinkedIn and upload them on arXiv as well, so I don't think reach is the problem. I work in the field of networks and security.

Does this mean that my work is not worthy enough? Or I am missing something in attracting the research community to cite my work?

  • 12
    I think you have two choices. Do the work you like, regardless of how few citations it garners. Or: switch to a hot topic, and do work with the citations in mind, ignoring how much you like or don't like that area. – GEdgar May 30 '17 at 1:15
  • 30
    Perhaps the problem is because you only publish in journals? As I understand it, for the field 'networks and security', conferences are more important. – Prof. Santa Claus May 30 '17 at 6:32
  • 3
    @GEdgar, even if citations are not a goal per se, a lot of us want to do work that really makes a difference, and citations should be correlated to that somewhat. – user24098 May 30 '17 at 11:14
  • @MBK What is the typical citation rate of other articles in the same publications you are publishing in? It strikes me that a raw X citations per Y publications is meaningless without a benchmark of publication and time. You may already be doing significantly better than average. – Myles May 30 '17 at 17:24
  • 2
    Bear in mind that some/many authors that get a lot of citations, exceeding a journal's impact factor (and therefore boosting it), get them by default just for who they are. I have seen many cases where groups of people just cite each other, regardless of quality, over and over. It's almost like an informal citing coalition (i.e., one that has emerged rather than being explicitly agreed). Likewise, the most interesting papers I read sometimes get cited few times. If you co-author with the right people, or you become better known, things may change. – Dr. Thomas C. King May 31 '17 at 9:24
49

You might want to calibrate your expectations.

What predicts how many citations you should expect?

  • Journal impact factor
  • Time since publication
  • Number of publications
  • Database used for counting citations

In rough terms, the journal impact factor can give you an idea of how many citations you can expect to receive per year for a given article. Note that this is based on the relevant database. Google Scholar citation counts tend to be two or three times larger from my experience. Note also that the distribution of article citation counts are highly skewed. So many will get fewer citations, but hopefully as you aggregate over a body of work, central limit theorem will kick in. So it's a reasonable guide to a benchmark how many citations you should expect over the body of your work.

Note also, the time it takes people to build on your work. So if you publish something late in 2014, it will often take at least a year for people to notice and start incorporating into journal submissions and for those articles to be published.

The basic consequence of this is that if all your work has come out in the last 3 years, and therefore much of your work has come out in the last year, then you may have to wait a few years to see the citations accrue.

How to calculate expected citations

So let's do some basic math.

  • Mean impact factor: Based on your stated quartile impact factors, I'm going to guess that your average impact factor per article is around 0.8 (e.g., top quartile is around 2 or 3+; second quartile is around 1.5; third quartile is around 0.5, based on discipline it could be different).
  • Mean time since publication: I'll guess 1.5 years, i.e., half-way between 0 and 3 years.
  • Discounting self-citations: Impact factor does not discount self-citations, but you have. So let's say 20% of citations are self-citations, especially early on in an article's life.
  • Number of articles: you say 17

So a very rough estimate of expected number of citations is:

expected citations = mean IF * mean time since publication *
                       self-citation discount * number of articles
 = 0.8 * 1.5 * 0.8 * 17
 = 16.32 (i.e., 16 citations)

A few caveats:

  • If you are using Google Scholar as your metric, then you should multiply expectations by a factor 2 or 3 (so that gives you an expected citation count somewhere around 32 and 39).
  • You could get a more refined estimate if you calculated expectations for each article separately and then summed the expectations.
  • There is quite a bit of uncertainty about the timing issue. For example, with impact factor, the first few months often don't count and sometimes differences between accepted versus online access versus published with page numbers might make a difference.

So actually, that does not look that different from what you have.

Also, note that in theory, you could stop doing research today, and the formula would predict that 15 years, you could expect about 160 citations. As with all things, this depends on various assumptions. But if you are comparing your citation count to senior researchers who have been publishing for 20 years, then you need to really understand the fundamental role of time passing in generating citations.

More generally, citation counts per article and even on aggregate can be quite noisy and the underlying distribution can be heavily skewed. So the difference between being above or below expectations may be whether you have one or two articles that have really taken off in terms of citations.

Take home messages

  • If you are a PhD student or early career research where the vast majority of your work has come out in the last two or three years, citations will generally look small. Citations just take time to accrue. And the number of citations an article accrue per year is quite noisy.
  • In the first couple of years after a publication comes out, the impact factor is probably a better predictor of how many citations it is likely to accrue per year.
  • Based on the assumptions above, publishing in an impact factor 3 journal will on average generate 3 times as many citations as an impact factor 1 article, and 6 times as many citations as impact factor 0.5 article. While this is all an on-average thing, the point is that it is quite possible that your your two Q1 publications are equivalent or more important than your 10 Q3 publications, from a citation generation perspective. So if you view things from an impact/citation perspective, then this can inform your understanding of the quality versus quantity trade-off when publishing your work.

Generating more citations

That said, if your goal is to generate citations from others, there are all sorts of things that you could be doing.

  • Try to do great work and publish in high impact publications. When evaluating a quantity versus quality trade-off, give a lot of weight to quality/impact.
  • Advertise your work at conferences and online.
  • Attend conferences and network so that people get to know who you are and what you are doing.

Other options that may or may not go into grey territory:

  • Self-cite where appropriate. Some citations will come from others seeing how your work is being cited. By self-citing, you are providing a template, and further secondary exposure to your work.
  • Collaborate and co-author papers with leading figures in your field. When they publish without you, they may cite your co-authored work.
  • Think about what topics receive more citations and do work on that. Important review articles, meta-analyses, methodological papers with a clear recommendation, etc.
15

Do you go to conferences? Giving a good talk is a great opportunity to advertise your work and get it known by a larger circle of researchers.

Also, I don't know where those download numbers come from, but don't take them too seriously. Typically these digits are inflated by search engine crawlers and other automated downloads.

  • 3
    One could contextualize the download numbers by comparing them to the download numbers of highly-cited papers. Not clear that's worth the effort, though. – Neal May 30 '17 at 12:30
5

This is probably normal.

Citation analysis has created a small storm in the late 2000's by showing that up to 90% of academic papers are not cited by anyone. This figure however only holds true for social sciences, since in other fields the figure goes down to 50% for most or even lower such as 18% for medical papers.

To go further, you can look at the distribution of the number of citations for a particular journal, and the median number of citations, instead of the mean, to get an idea of the citation repartition.

Citations distributions per journal: Citations distributions per journal

Median citations per journal Median citations per journal

Figures from Quantixed article: The Great Curve II: Citation distributions and reverse engineering the JIF.

These figures were generated on 2014 for articles published in 2012 and 2013 in the same journal.

If you look at high impact factor journals, such as the venerable Nature, you can observe that the distribution is highly skewed: most articles only get 0, 1 or 2 citations, not more (at least over the timeframe of 2 years since publication). Thus, publishing in high impact factor journals does not guarantee citations, it just increases the likelihood.

This also shows that high impact factor does not equate high number of citations: a high impact factor is a measure of popularity, in other terms a measure of how many people have read it. You might potentially get more citations by publishing in journals that are more focused on your topic, even though the userbase is smaller (smaller impact factor). This is something to keep in balance (journal scope vs impact factor).

Why are citations propagation so slow? Contrary to journalistic, or social media, posts that get propagated a lot faster, here the slow pace of citation increase can be explained by various factors (that are still being researched), but without doubt two factors are proeminent:

  1. Academic articles are composed of long and dense information to read. It's not just something you glance a few seconds/minutes and can decide to share, reading academic articles sufficiently to understand them takes much longer.
  2. Citing is not sharing (it's more): citations are an estimation of the number of "derived works", whereas sharing is just copying or creating a link. Thus, citing is much more involving than just sharing, since to cite you need to write your own paper and get it published. This is why some research tools offer alternative metrics such as Mendeley reader counts.

There is no clear solution to the "citation issue" right now, so that's why in my opinion the most important is to enjoy research and to be proud of the work you accomplished with your publications.

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.