31

I am quite surprised to find that a lecture slide for a course has 133 citations from highly reputable researchers across the world. The main contribution of the lecture slides (which seems to be the reason for the citations) appears in 1 line on 1 page of the slide out of 30 slides. (Citation count according to Google Scholar)

What is the point of even publishing a paper and going through the painstaking process of peer review and editing if you can just write some blog post or a lecture slide on some hot topic and accumulate citation counts (which is crucial for securing funding, etc.)?

  • 21
    For the peer review ? – onurcanbektas Feb 27 at 17:52
  • 26
    "citation count" from Google Scholar is not a real citation count. It is automatically compiled, includes lots of false positives, duplicates, etc. – Morgan Rodgers Feb 27 at 19:08
  • 2
    The real question is whether journal articles are the best or only way of communicating on research, depending on what you want to say and to whom. I do not see the point of framing the question in terms of citations, though. – Sylvain Ribault Feb 27 at 20:55
  • 1
    @smci In my experience, if one of my publications gets two duplicate entries I can merge them; but if one citation gets counted as a citation against the wrong paper, or something like that, I've never noticed it getting fixed. – Morgan Rodgers Feb 27 at 23:39
  • 1
    @smci Haha no if you have two entries corresponding to the same article they have to share citations, the citations don't count for both unfortunately. – Morgan Rodgers Feb 27 at 23:46
51

It's not fair to only look at the peak of the distribution. For an apples-to-apples comparison, you need to compare peaks to peaks and averages to averages.

The two sources you mention are both in the field of machine learning. If we assume that they correspond to the blog/lecture notes sources with the most number of citations (i.e. the peaks), then we can conclude that these venues can generate at most ~700 citations. If you compare to the most cited machine learning papers, these 700 citations are minute. For example putting "machine learning" into Google Scholar yields:

Scikit-learn: Machine learning in Python (journal article) -- 14919 citations

Data Mining: Practical machine learning tools and techniques (book) -- 34724 citations

What about averages? I don't know what the average number of citations a blog post or lecture slide gets, but I'd guess less than one, since many blog posts don't attract comments. The average number of citations for a journal article however is easy to find - just look at the impact factor. Putting in "machine learning journal" into Bing, I get journals such as Machine Learning (IF = 1.855 as of time of writing) and International Journal of Neural Systems (IF = 4.58). Clearly the average journal article gets a lot more citations than the average blog post or lecture slide.

tl; dr: What's the point of studying machine learning instead of playing soccer and earning hundreds of thousands of dollars a week? The answer to that question is similar to the answer to this one.

  • 12
    Scikit-learn is a software library, the vast majority of those citations are for people using the software, not reading the paper. Likewise, the second example is a book. Hardly an apples-to-apples comparison with the expected citations for a journal paper (although the top journal articles can get thousands of citations). – user2699 Feb 27 at 17:32
  • 4
    Machine learning is less likely to lead to permanent brain injuries, unless you facepalm too hard. – darij grinberg Feb 27 at 20:23
  • 7
    -1: This is not a technical question about citations. The real issue is the dominance of journal articles in scientific communication. – Sylvain Ribault Feb 27 at 20:58
17
  1. The instances you mention don't show that statistically blogs or slides get cited well. Just that you find some instances in the universe of events.

  2. Papers tend to be abstracted (chemistry does this quite well). Blogs and slides not.

  3. Science citation searches don't generally index blogs and slides (especially uncited ones). Thus they can be difficult to find during a lit search.

  4. Journals exercise a function of review and editing that drives a superior work product in formatting. Blogs and slides are generally a mess in their referencing fro instance, compared to papers. It's not just that editors and reviewers drive this but that authors tend to "up their game" when sending work product for review.

  5. There is some benefit in review scientifically (more so for weaker papers, but still).

  6. Papers help your career.

  7. Nothing prevents publicizing a paper by blogging or presenting it in addition. And usually the blogging or presenting will be superior because solid work has already been done previously.*

  8. Narrative technical reports ("Word documents" or the sort) are generally superior to slideware in information density and quality. [Read the Tufte contributions to the Space Shuttle disaster inquiry for some of this...Feynman had same issue with the previous disaster and the problems with slides versus sentence and paragraph reports.

*Small aenecdote to explain. I took a course once where we had a true seminar (oval table discussion with small group) on foreign policy controversies. Every Tuesday, we handed in a 2 page written paper before the discussion (on a set of readings). Every Thursday, we just had discussion, on a new set of reading, but no paper was required. The Tuesday discussions were stunningly better than the Thursday discussions. Doesn't this make sense when you think how much better you understand something after writing it up properly? The same applies for doing a presentation or a blog on a piece of science (after writing it up formally).

  • 1
    6. Papers help your career. – eric_kernfeld Feb 28 at 13:23
11

Slides and blog posts do not get more citations than journal papers.

You've given non-representative or invalid examples:

  1. For the paper following a blog post: It's the ArXiv paper that has the citations, not the blog post.
  2. For Andrew Ng's course notes: Sometimes, professors arrange for a course-note-taking semesterial project which is intended to eventually produce a textbook. When this happens, notes are taken much more seriously, beefed up a lot of after class, go through round(s) of review by the professor or peer students, follow strict formatting guidelines etc. This is nothing like lecture slides. Also, textbooks != papers.
  3. The "Notes on Convolutional Neural Networks" document is not a deck of lecture slides nor a blog post. The fact that it has "notes" in the title does not mean it is just some scribbles jotted down - it looks serious.

... so your premise is unsubstantiated. Actually, it's incorrect as far as a I know.

7

Apart from the overgeneralization in your premise, which was pointed out in other answers:

Blog posts and lecture slides don't carry the scientific authority of peer-reviewed and professionally edited publications like journals. Peer-review and the services of scientific editors serve as a vetting process to filter out publications that don't hold up to scientific scrutiny. The vetting is performed by (usually two or three) qualified researchers on behalf of the scientific community. By contrast, the credibility of a blog post or lecture slide depends entirely on the reputation and trustworthiness of its author. Not only is this a less stringent criterion, it is also hard to assess for outsiders.

5

The principal contemporary reason for formally publishing articles is that governments and institutional administrators demand of researchers proof of their productivity. Being unable to assess such productivity according to their own criteria (in general because they have none) they attach simple metrics to research activity which they use to rank researchers. The principal metrics are money secured in competitive grant programs and counts of papers indexed by some supposed authority. In the current moment, publishing in journals serves mainly to achieve the second goal.

Formal journal publishing generally adds little value from an intellectual point of view and generates obstacles (paywalls) to dissemination of ideas. Something like the ArXiv achieves wide, free, dissemination of knowledge in a rapid and easy way. The author can write an article according to the author's criteria and distribute it as the author sees fit. Sometimes the refereeing process adds value, when the referees and editors behave in a serious fashion, but more often they do not or the process simply delays dissemination.

  • Yup. In science, no refereed paper leads with certainty to no grant. – ZeroTheHero Feb 27 at 12:30
  • 3
    Peer review may not add value to every individual paper, perhaps not even to most papers; but it does add value to the credibility and validity of scientific publications on a systemic level, namely by filtering out at least some of the work that doesn't hold up to scrutiny. – henning Feb 27 at 12:53
  • 1
    "Sometimes the refereeing process adds value, when the referees and editors behave in a serious fashion, but more often they do not" - do you have any data to back this rather sweeping claim up? Yes, there are problematic editors and reviewers, but in my experience across two unrelated fields, it is simply untrue that this occurs "more often" than that they are helpful. – Stephan Kolassa Feb 27 at 14:21
  • @StephanKolassa: "but more often ... or the process simply delays dissemination". By quoting in such a way as to omit the second part of an "A or B", you have perturbed the intended meaning. I make no "sweeping claim". In any case, data about such secretive processes as reviewing is not readily available. I see very little evidence that peer reviewed articles are "better" than those distributed via the ArXiv (for example). Certainly they are less accesible and take longer to reach their intended audience. – Dan Fox Feb 28 at 8:46
  • 1
    the process simply delays dissemination — The peer-review process can only delay dissemination in fields that actively prevent faster means of dissemination (like conference presentations, technical reports, preprints, working papers, and the like). – JeffE Feb 28 at 10:03
4

The "citation count" from Google Scholar is not a real citation count. It is automatically compiled, includes lots of false positives, duplicates, etc.

Also it counts citations from things like other lecture slides, blogs, etc. None of these will count for anything when you are applying for funding. Any sort of funding/hiring/promotion that wants to look at your citation count is only going to count:

  1. Citations of your peer reviewed publications
  2. from publications which are themselves peer reviewed.
  • 2
    [citation needed] [citation needed] [citation needed] This answer is utterly inconsistent with my experience in computer science, which does take Google Scholar seriously, and which does "count" citations both to and from unrefereed papers (and, yes, even blog posts) in hiring, promotion, and funding decisions. – JeffE Feb 28 at 10:01
  • 1
    @JeffE: This may vary quite a bit by subfield and institution. My impression (from computer science, although oriented towards the applied end, and in Germany) indeed matches more or less the perception described in this answer, in that Google Scholar's citation counts should, at best, be used with extreme caution and are, at worst, little more than a joke. Issues like undergraduate seminar write-ups listed as doctoral theses or your citation count getting hugely inflated by counting all kinds of bachelor theses or coursework that you happened to supervise seemed commonplace. ... – O. R. Mapper Feb 28 at 13:11
  • ... As you have made quite a different experience, maybe the accuracy of Google Scholar's citation count (or its complete indexing) differs greatly depending on how well the indexing software gets along with different data sources. – O. R. Mapper Feb 28 at 13:11
  • @JeffE my experience (also from CS) is that the institutions want to outsource the "taking it seriously" evaluation to someone else, so my local institutions refer to scopus or web of science for evaluating which publications matter, what are the impact factors and counting citations, so unless both my paper and the citing paper are indexed by one of those, then they might as well not exist from the evaluation perspective. – Peteris Mar 1 at 1:42
  • 1
    @MorganRodgers We don’t use the citation counts for anything but crude, high-level comparisons between people of similar academic ages in the same subsubfield (because citation patterns very too much to sensibly use it for anything else). We also ask experts to evaluate the content (reference letters) and (whenever possible) we also read the papers ourselves. – JeffE Mar 1 at 9:16
3

Is your primary goal really citations? Or is your primary goal to contribute in a substantive way to the research in your field?

In my fields (Physics & Engineering) journals articles are where the details of new research can be found, and are accessible for decades or centuries. It is not a rare to find important information in older published works. This is how the knowledge of human kind has been growing for centuries, and as a method of structuring academic information sharing, it works.

Even more, if you want to contribute something to this world, journal articles and the peer review structure have the permanence to allow your ideas and contribution to connect with others. It is true that newer media methods also can do this, but journal articles are a very concrete way to contribute for the long term.

However, if you do not want to contribute to the grand project of increasing human knowledge, and are instead interested only in increasing your citation count, why bother with peer review?

  • 1
    In my fields (Physics & Engineering) journals articles are where the details of new research can be found, and are accessible for decades or centuries — I think you misspelled "arXiv". – JeffE Feb 28 at 10:05
2

Journal publishing is about branding. There are three ways to create a successful mousetrap:

(a) Design the best mousetrap in the world and hope that the news spreads by word of mouth. If it's really outstanding, then you might succeed.

(b) Establish a reputation as the best designer of household gadgets in the world, so that people are actively looking out for your next idea.

(c) Advertise your mousetrap in a catalog produced by a well-known retailer, so that a proportion of people will buy it even it it's nothing special.

Exactly the same is true if you're a researcher peddling intellectual ideas. A very small number of people are capable of succeeding using approach (a) or (b). For the rest, the only way to get anywhere is approach (c) - where the retailer/brand in this case is the journal that you publish in.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.