Is there an open source tool for producing bibtex entries from paper PDFs? [closed]

Question

In the desktop application for Mendeley, it's possible to import a PDF and have the bibliographic data automatically extracted (or perhaps looked up, I'm not sure). This feature is more or less reliable. However, Mendeley's code is closed source (they once promised to release the source, but then they were bought by Elsevier). I'm looking for an open source tool that takes one or more PDFs as input and returns a bibtex entry for each.

I've found the following, but couldn't get either of them to work:

• cb2Bib
• pdfmeat

At present, the fastest alternative I know is to copy/paste the title into Google Scholar, and click the link to bibtex. That's very nice, but I'm wondering if there is something more automated.

Answer 1

I use Zotero which in itself is a system for handling references, it comes as both a plugin to Firefox and as standalone. I use the standalone version to extract reference information from pdf and then export to, in my case, BibTeX .bib format. There are possibilities to export to other formats as well.

Answer 2

This doesn't answer your entire question, but may be useful (for example, you might have got the papers from a list of DOIs in the first place).

Assuming these are PDFs with CrossRef DOIs, if you can extract the DOI from the PDF, you can get citation directly from CrossRef's API. For the DOI 10.5555/12345678, the query:

http://api.crossref.org/works/10.5555/12345678/transform/application/x-bibtex

returns

@article{Carberry_2008,
    doi = {10.5555/12345678},
    url = {http://dx.doi.org/10.5555/12345678},
    year = 2008,
    month = {aug},
    publisher = {{CrossRef}},
    volume = {5},
    number = {11},
    pages = {1--3},
    author = {Josiah Carberry},
    title = {Toward a Unified Theory of High-Energy Metaphysics: Silly String Theory},
    journal = {Journal of Psychoceramics}
}

You could write a very small script to scan a list of DOIs and download the citations.

Answer 3

NB: My answer does not differentiate between open and closed sourced projects and I have not used any of the seemingly big list of solutions.

This SO answer suggests that the 2010 London Dev8D meeting, whatever that is, ran a contest for meta data extraction and resulted in pdfssa4met. I cannot find any documentation on the meeting and anything else that came out of it. The JISC ConnectedWorks project produced a review document that considered Zotero, Mendeley, Google Scholar, CB2BIB, Metadata Extraction Tool, pdfssa4met, pdfmeat, GNU libextractor, FITS, Apache Tika, XPDF, PDFTOHTML, pdf2xml, CiteSeerX, and Paperpile. This list seems to leave out some other solutions, although it is possible that they rely on the same underlying technology. This answers to this TeX.SX question suggests BibDesk and JabRef do metadata extraction. Papers also seems to do metadata extraction. This blog reviews the metadata extraction performance of WizFolio.

There is also Mr. dLib, pdfextract and TeamBeam which seem to have scholarly papers associated with them and therefore seem to be misssed by the JISC review (or developed afterwards). I also found exiftool.

Answer 4

cb2Bib is a tool to extract bibtex entries from PDF files.

The following will command extract bibtex entries from PDF file using cb2Bib command line

c2bconsole --doc2bib paper2.pdf references.bib --sloppy

Answer 5

Disclaimer: This is a short version of an answer posted at tex.sx. This solution is not perfect, but might be a good start. I am one of the authors of JabRef and like open source development.

JabRef is an MIT-licensed open-source BibTeX and BibLaTeX bibliographic manager actively developed on GitHub. It offers the functionality to import bibliographic data from PDFs.

1. Create or open a .bib file.

2. Go to "Quality" -> "Find unlinked files".

3. The "Find unlinked files" dialog opens.

4. Choose a directory using the "Browse" button.

5. Click on "Scan directory".

6. In "Select files", the files not yet contained in the database are shown.

7. To create entries for all files, click on "Apply".

8. For each file, an import dialog is shown

   The dialog shows the XMP metadata stored in the PDF in the area "XMP-metadata". If this data fits your needs, select "Create entry based on XMP data". Typically, the XMP-metadata is not good enough. Choose "Create entry based on content".

9. Click on "OK" to start the import

10. A dialog asking for the link is opened You can choose "Leave file in its current directory" to keep the file where it is. Typically, this is that what one wants. In case you choose "Move file to file directory", you can also choose to rename the file to the generated BibTeX key.

11. Press OK to link the file to the BibTeX entry

12. This happens for each file. After that, the "Find unlinked files" dialog is shown. Just click on "Close" to close it.

Answer 6

You might want to look at pdf-extract: https://github.com/CrossRef/pdfextract

It doesn't seem to be very actively maintained, but promises to do what you want.