Is there documentation on how long it takes before publications are indexed by Scopus and Web of Science?

Question

The two major bibliographic databases - Scopus and Web of Science\Web of Knowledge - have one very badly-documented limitation; their update lag. While the majority of papers are added quickly, it can take several months for individual papers to be added.

As an example, I ran a search in Web of Science looking for all papers published by my institute in 2014. In November 2015, it reported 290 papers. In January 2016, 293. In February, 305.

In other words, a few percent of the papers took a year or more to be added to the database, which startled me - I was expecting a lag of perhaps three months. Some of these delayed entries are from relatively obscure journals (specialist Australian publications) or book series, and it seems reasonable that these might take longer, but most are from mainstream titles - one is even from Philosophical Transactions of the Royal Society, as established a journal as it's possible to get.

Scopus is comparable. In November 2015, it reported 340 2014 papers, 356 2013 ones and 351 2012. Running an identical search three months on brings us to 341, 367 (!), and 353 respectively - so a few papers are still being added up to three years after publication. (Unfortunately, Scopus doesn't expose the date items added to the database, so it's not possible to identify which ones were added most recently - it's possible that some were due to journals being added to Scopus for the first time along with their back-issues.)

This is, of course, something of a headache for producing stats. But more practically, it could be a real problem for disseminating science - something being unintentionally missed or delayed from one of the big indexing services can make it much less discoverable and much less likely to be read, built upon, or cited, especially in that important first year.

So, the core question: does anyone know of documentation (or research) into how substantial these delays are; how they're distributed; and whether they're getting better or worse?

(I've spent some hours searching for this but to no avail. I'm wondering if I may just be looking in all the wrong places.)

Answer 1

This is a really interesting question!!

According to Clairvate Analytics WebofScience/WebofKnowledge), updating of the WOS collection occurs daily on Monday through Friday. This revelation, paired with @Andrew 's comment about Scopus, suggests that the lag time exists with the publisher and not the database compilers (i.e., how long before the publisher submits (or makes available) the DOI and metadata). Here is an interesting op-ed on these lag times.

It is also important to note that articles which are published in non-indexed journal will not appear in most collections/databases, however, GoogleScholar will may pick up more grey literature (e.g., government articles, theses), and potentially some of the non-indexed journals. This article suggests provides 'optimal' updating frequency compared to GS and WOS in their field.

If the article is available online as either an early print or post-print, then it should have a DOI (I assume), and therefore should be searchable via Google. It follows that Google Scholar should pick up articles faster than human-curated databases (Scopus, WOS).

In a brief search (using GoogleScholar), I came across a preprint by Haustein et al. When is an article actually published? An analysis of online availability, publication, and indexation dates.. The authors found that in some fields lag time has decreased but has increased (!) in others:

Various studies have analyzed publication delays and found differences between scientific fields, journals, and publishers (e.g., Abt, 1992; Amat, 2008; Björk & Solomon, 2013; Das & Das, 2006; Diospatonyi, Horvai, & Braun, 2001; Dong et al., 2006). Since long delays interfere with priority claims and slow down scientific discourse, publication speed plays an important role for authors and scholarly communication (Rowlands & Nicholas, 2006; Schauder, 1994; Tenopir & King, 2000). Short publication delays can therefore be considered as a quality indicator reflecting the up-to-dateness of scientific journals (Haustein, 2012). Publishers have begun to reduce delays by making so-called early view, in press, ahead of print or online first versions of accepted papers available before they appear in an (print) issue. It has been shown for food research journals that online ahead of print publication has reduced publication delay by 29% (Amat, 2008), while Das and Das (2006) reported for 127 journals in 2005 average lags of three months between online and print issues publications with particular differences between publishers. Tort, Targino, and Amaral (2012) showed that this lag increased significantly over time for six neuroscience journals. Online dates are now being recorded in bibliometric databases like Scopus, which impacts bibliometric analyses (Gorraiz, Gumpenberger, & Schlögl, 2014; Heneberg, 2013).

Various distributions are made available in Table 2 of the aforementioned paper.

An aside: it may be worth checking out some of the works citing Haustein et al.