The Ethnography Of Obscure References

Question

I want to understand how absurdly obscure references are discovered by authors when preparing papers for publication.

Although the availability of large online searchable databases obviously makes things easier today, I still occasionally come across a paper or a book, sometimes from decades ago, which somehow discovered a paper or letter to the journal editor, written 50 years ago, in another country, in a foreign language, in an obscure long gone journal, from an unrelated field, which deals with a highly specialized problem, that has a shockingly generic title, and presents not the same idea, but one which can be shown to be equivalent to it, in a brief parenthetical comment, using defunct jargon, dead nomenclature and outdated notation.

Although you don't often find strictly all of the above to be true at the same time, some generous subset of it does occur more often than one would expect.

How are these references discovered? Is there an army of science historians scouring agricultural journal from 19th century Prussia for proto-theories of quantum entanglement? Does every academic department dedicate a portion of its budget for the care and feeding of some tiny Hungarian septuagenarian savant with eclectic tastes and photographic memory? Or is the reference librarian a vastly unappreciated form of superhero?

Answer 1

Update #0: Case Study: Logistic Regression

As an example, consider the origins of the logistic equation as a model for population growth. I came across "The origins of logistic regression" , Cramer 2002 on wikipedia in the context of machine learning. Cramer traces history back to the independent rediscovery of the equation by Pearl (Bio statistician at Johns Hopkins) and Reed in the 1920s which appeared in the "Proceedings of the National Academy of Sciences", in "The Quarterly Review of Biology" and others. Cramer notes that the name Velhurst is first mentioned in a footnote in one of their papers which he cites by year but does not appear in the bibiliography (Pearl and Reed 1923), and on page 560 of pearl's 1924 book Studies In Human Biology we get the following:

(This criticism distorts the facts, according to Cramer).

Pearl and reed cite Verhulst which published, in french, in the 1844 proceedings of the Bruxelles Royal Academy, but actually earlier in 183/8 in a journal published by his teacher Quetelet (Who Developed the Body Mass Index], in the Netherlands.

Du Pasquier cited this work in a paper published in 1918, 74 years later after Verhulst, in The " Vierteljahrschr der Naturforsch" ("Nature Studies Quarterly"?) which was apperently a german general published in Zurich, though Du Pasquier's paper on population models, "Esquisse d’une nouvelle theorie de la population" was in French. How did Du Pasquier learn of this 75 year old paper? According to WP he studies in both France and Switzerland, and was not only a mathematician but also "a historian of mathematics and mathematical sciences" (!). It seems plausible that Reed and Pearl could have been aware of a paper published 2 years prior,even in foreign language and journal. And if not directly, their wider circle could have turned their attention to it. Though, reading the quote, they seem to possibly be saying that they reached the Verhulst paper by a different and unspecified route.

So, in this case, a polyglot European mathematician with a passion for math history, discovered (again, in 1918, from a foreign journal) an obscure 80-year old work, cited it in his own paper, which was enough to get it over the ocean to america two years later, where the same ideas had been redisovered, but much more effectively publicized. With that, the citation link was forged, and now we have the Pearl-Verhulst logistic process.

One conclusion is that you don't necessarily need someone who knows the obscure paper, you just need to transitively traverse the citation graph and hope that someone cited it at some point, and that you hit the natural limits of exponential growth before you stop looking.

Update #1: Before Information Overload

a suggestion received from an unexpected source. In reading the 1988 article "What's Wrong With This Lagrangean?" by David Mermin, he suggests a fact which explains how knowledge of sources used to be work:

We've known for some time that, roughly speaking, nobody any longer reads anything but preprints, the archival journal of choice, which for many years now has been Physical Review Letters, and secondary references cited in these two primary sources. But the preprints have been coming thicker and faster. And Physical Review Letters now publishes almost as many pages each month as all of Physical Review did back in 1956, when I was starting graduate school.

[...]

the disaster looming over science libraries, and therefore over science itself, as a result of the irresponsible way we have allowed scientific journals to proliferate; and, not unrelatedly, the lamentable decline in the quality of scientific writing.

In other words, a hundred years ago, before volume became unmanageable, people actually used to read what was published.

Update #2: Memory In Judaism

Regrading the role of persons possessing exceptional memory abilities, specifically in the scholarly context of judaism, here is an extract from "The Esential Talmud", Ch.6, by Adin Steinsaltz:

The work of preserving and codifying the vast body of oral law went on for several generations, but its importance waned as the main focus of Torah scholarship shifted elsewhere. Those scholars who engaged in memorization of the vast number of baraitot were still known as tannaim, but the term took on new significance. Tannaim were no longer creators of the oral law but individuals gifted with phenomenal memories who did not always comprehend the full significance of what they were memorizing. They were to be found for many generations in the larger academies, serving as living archives utilized by the sages in order to clarify and elucidate various problems that cropped up in the course of the study of the Mishnah.

See also the related, tragic, short story "Mendel the Bibliophile" (1929) by Stephen Zweig. These persons are rare, but have played an important role in preserving knowledge throughout history. I've ran into more than one, behind the counter at this or that second-hand bookshop.