Should I keep all my .bib files in a single folder, or one with each .tex file?

Question

Self-explanatory. I would be most interested to hear from researchers who have published a fair amount, decided one way or the other wasn't working for them, so switched — and why.

Related advice / observations welcome: e.g. "why the hell would you need more than one .bib file?"

Answer 1

• Several .bib files in the a single folder:

You might want to do this if you write about quite disjoint topics, or if you want to keep several sets of inherently different references in separate files (e.g. scientific publications in one file, technical standard documents in another, etc.). Overall, however, I see little reason to choose this approach.

• One .bib file (obviously in one folder):

This allows you to build up a database with your personal literature collection. Given that BibTeX by default only shows cited references, this is one of the ways the system is supposed to be used. For someone working entirely alone, this might be a viable way to go.

• Several .bib files in several folders:

This is the approach I follow (using JabRef), with the further restriction that the .bib files usually reside in precisely the folders where they are used by .tex documents.

Advantages:

• The .bib file is a part of the source. When using a VCS, everything required to build the document should be in the VCS, and with one repository per paper/project, the appropriate .bib file needs to be stored in each repository.
  • This might be solvable by including repositories in repositories (such as with SVN externals), but that still assumes a central repository location that is accessible to all co-authors, which is not a given when collaborating with different groups.
• As also remarked by Federico Poloni, when several authors work together, they need to use the same references. It wouldn't make any sense if each author had his or her personal large .bib file, rather than having one common .bib file for the paper/project.
• Even when working as a single author, the .bib file sometimes needs to be submitted for the camera-ready version of a paper, to allow editors to build the document themselves. While I don't think it's usually explicitly forbidden, I'm quite sure it's not a very good idea to submit your complete multi-MB literature database every time you submit a CR version.
• Different papers/projects need to be formatted according to different styleguides. While the actual layout of the bibliography is imposed by the BibTeX Style that usually comes with the paper template, some paper-specific tweaking is required more often than not in my experience:
  • Some styles show URLs, for other styles, the URL needs to be inserted into the howpublished or the comment field.
  • In some papers, you want to (or have the space to) show some redundant information such as publication months, publisher locations, or DOIs, in others, you don't.
  • In some papers, you may want to use full journal or conference proceedings names, in others, you may want to abbreviate them as far as possible
  • In some papers, you can use special packages (e.g. for correctly rendering a Latvian name with a comma accent), in others, this might not be allowed.
  • In some papers, the layout of the bibliography is such that you need to repair some ugly block formatting with additional \hskip commands, custom hyphenation, and the like.
• When starting a new paper or project related to a topic I had written about before and want to grab some random related work for the introduction, I find it most convenient to open the .bib file of the previous document to get an overview of the ~30 references that I had used there. Of course, I could also look in the compiled PDF file, but I cannot directly copy the entries that seem suitable from there.
• As mentioned by darij grinberg, it might not be desirable to have later changes to bibliography items retroactively show up in old documents. It would mean that the old sources compile to something else than what they did at the time of writing the document, and it may even ruin a carefully adjusted layout.

Disadvantages:

• I regularly need to copy some references from one file to another when I want to reuse them.

Answer 2

I have a single Mendeley-generated bib file (1.8MB as of now).

There reason in the following:

• Mendeley works that way,
• it is convenient to be able to add a bib entry just once, so it can be easily used in any paper.

However, it has its problems as well, especially related to collaborating with others, and encapsulating content:

• any changes in a bib file (e.g. modifying existing entries, errors) will affect all files,
• it's harder to add citations from collaborators without affecting your structure (especially as citations from collaborators can have different key conventions),
• if it need to be in some git repository, it requires a copy of bib file, with many non-relevant entries.

In any case, when collaborating with others, or finalizing a project, I do copy the whole bib file to protect myself against changes.

I did try to extract a subset of citations, which I use in a given paper, with bibexport (learnt from Creating .bib file containing only the cited references of a bigger .bib file - TeX.SE), but with some various results (it seems that bibexport does not support all fields supported in Mendeley); but it was some time ago, so maybe things are different.

See also:

• Strategy for managing multiple Bibtex files - TeX.SE