It is encouraged to cite relevant software in research papers (e.g. MIT guide). The question is,
Are these citations indexed at all by any engine?
I can hardly find any software on e.g. google scholar to navigate the citations. For example, SageMath devs request to notify them manually about each citation to collect them on their page.
I am of course talking about software without an accompanying research paper.
If by indexing you mean, "are citations for a particular piece of software trackable", the answer is "yes, they can be".
The key to indexing is to give your piece of data - software, papers, whatever - a persistent unique identifier. This identifier is traceable back to a particular version of something, e.g., a specific data file or a particular release of the code, or a corrected paper. One example of this unique identifier is a Digital Object Identifier or DOI.
Until recently, typically only journal papers published by larger journals were given a DOI. In the last few years organisations such as Zenodo.org have started to set up the infrastructure to assign DOIs to other digital works, including software. This means that any piece of software can have a DOI and is potentially indexed. See e.g., this search on Zenodo for a list of software.
From that list of software I clicked on a totally arbitrary result that I'll use for the rest of this answer.
The DOI of the software is 10.5281/zenodo.2585902. This DOI can be resolved using the https://doi.org/ service. Or, I can link to https://doi.org/10.5281/zenodo.2585902 and I will be redirected to the source code immediately.
Once a digital object has a DOI, it can be cited.
On the Zenodo page for my example software I can download a citation in a Bibtex format:
@software{tanlabcode_2019_2585902,
author = {tanlabcode and
software-github},
title = {software-github/SCRABBLE: make it suitable on CRAN},
month = mar,
year = 2019,
publisher = {Zenodo},
version = {0.0.1.1},
doi = {10.5281/zenodo.2585902},
url = {https://doi.org/10.5281/zenodo.2585902}
}
Zenodo also generates citations in other formats like Mendeley, CiteULike, and others.
I can now include that citation in a paper. It is crucial that the reference list includes the DOI so that the citations can be tracked.
We should also note that citing the first research paper where a piece of software is mentioned is not the same as citing the code, especially as the code will change over time. The great thing about a DOI from Zenodo or JOSS is that you can cite a specific version of the code directly.
At the bottom of the Zenodo page for the software you’ll see a box listing citations for this publication. This is based on exchanging citation data (references to the DOI) with other services. Together these services cover much of the international publishing community and data repositories. It can take a week or two, but usually citations show up.
Another important tool is crossref.org which keeps track of all uses of a DOI. I can search the metadata index for that DOI. Unfortunately the random piece of software I chose has no citations yet :(
As mentioned by @moinmoin the process has been streamlined a bit by the Journal of Open Source Software (JOSS).
Although software and other digital objects can be indexed, this does not mean that they are. As pointed out before, Google Scholar is one of the few that seems to do this; Web of Science and others don't appear to include non-traditional DOIs at this time.
You should expect increasing indexing (and awareness) of non-article DOIs in the next few years as employers and authors see the benefit of being able to track non-journal publications, such as data sets, software, and other digital products. It gives everyone involved much more recognition than would be possible with traditional papers.
Let's look at your example, SageMath.
figshare-blah, and a series of DOI versions figshare-blah.v1, figshare-blah.v2, etc., with the generic resolving to the most recent. (I don't know, and it's not clear from above, how Zenodo's DOI versioning works.)
Google Scholar does sometimes parse citations to codes; here is an example of the R library rvest:
In addition, as MoinMoin implied, many researchers write so-called data papers or software reviews that accompany newly published datasets or softwares. They are submitted to journals and undergo peer-reviews like other kinds of scholarly articles.
Journals increasingly support such data papers or software reviews. I am not only talking about journals that are specifically dedicated to such document types (e.g. Springer Nature's Scientific Data), but also to specified submission guidelines for this kind of documents at other journals (e.g. at PLOS ONE)
Some bibliographic databases capture such documents with a special category. At Web of Science, for instance, you can refine the results to 'data papers', 'database reviews' or 'software reviews':
To an extent this is a known problem -"how do I get credit for writing scientific software?" when our community largely tracks citations via Scopus, ORCID, Google Scholar etc., and also when software is versioned and in continual development, not usually one-and-done like a paper.
I do not believe (read: have never seen) any kind if citation counting for the "miscellaneous" references (to borrow the common entry type used in BibTeX for software) without a proper identifier, like a DOI. I also anticipate that there's confusion in citation counts when trying to sum over versions of a software package.
Fortunately, being a known problem in scientific software development, a current attempt at a compromise solution that avoids writing an publishing a traditional, formal paper is The Journal of Open Source Software (JOSS). The JOSS papers are essentially taken from the documentation, so if you write up the code properly, it is supposed to be minimal effort to generate the paper. They then send it out for peer-review, and assign a DOI on acceptance.
JOSS also has the advantage of putting all the relevant links in one place at one place (DOI) - paper, source code, releases, etc. JOSS has the backing of NumFocus, who also have a few other open journals and are a major funder (donation bundler) for a large number of scientific, open-source software projects (Julia, Pandas, NumPy), which is a long way of saying that JOSS is a legitimate and accepted part of the Scientific FOSS ecosystem and not some scam journal.
