How to manage theorist's “labbook”?

Question

Question:

Are there any guidelines, sources, or examples of good practices for maintaining theorist's research notebook?

Motivation and context:

Maintaining a lab notebook is a much advised (if not a vital) tool for doing research. Many do that nowadays also in electronic form. While the methods for writing and maintaining a labbook might differ in details, in "dirty-hands" (no offense!) experimental, or field disciplines, the basics are relatively straightforward and clear. The advice tends to boil down to keep a labbook per single topic one works on and then record all the experiments and thoughts related to those experiments into the notebook. Simplifying a lot, in the end, the researcher (team) would compile a paper from the notes in the notebook.

Now situation of a researcher in a more theoretical discipline is strikingly different. Usually one engages in research on several topics at the same time. The problems a researcher works on are often intertwined and one's view on them changes over time, often merging originally separate streams of thought into a single one, or vice-versa. The work revolves not around experiments which would have a clear objective, but rather around currently prepared papers and then "a cloud of ideas". In summary, theorist's workstyle tends to be very non-linear. Maintaining some kind of a notebook is still a good thing, if for nothing else, then for not forgetting about stuff. My question is about how to go about doing it.

I am after a practical tool helping me to archive my thought process for long-term, as well as efficiently manage and keep an overview of the evolution of the several interacting topics I engage in.

Answer 1

(Supplementing F'x's answer.)

Physical notebook

I use it all the time, as it is the quickest way to write mathematics and jot diagrams. It's low tech, so it's "always on" (e.g. I cannot run out of batteries).

I have one A4 notebook (so I take it anywhere), with removable pages.

I try to devote a single page only to a single project. On each page I note 3 things:

• date,
• title (or, actually, a pictogram) of the project,
• "page number" for pages on the same project in a given day.

After some time (could be the same day, could be never) I take off pages, to fit them into thematic folders (and usually throwing away most of things, because they are "non-recyclable" rubbish; if it is "recyclable" then I scrap the important things, writing them on a new page).

However, such operation has trade-offs:

• (+) sorted by topic,
• (+) higher valuable content density,
• (+) can be viewed "all at once" by putting pages on a desktop,
• (-) I cannot take all of them anymore,
• (-) it's somehow easier to loose it, unsort, or torn/coffee split/... it.

Electronic notes

Electronic content is much more linkable, searchable, easier to share, more polished etc. However, it takes much more time to write formulas or draw diagrams.

Different things work for different people, but some kind of personal wiki (or a well-organized text file system) seems to work the best.

I'm not settled down, but I'm switching from TiddlyWiki (very neat but requires a browser to run) to Gitit (so I can write in Markdown, with LaTeX inclusions, in any text editor).

I tried Evernote as well (and I'm using it for other content) but for highly-linked content it does not work for me.

See also:

• How do you keep your research notes organized? - MathOverflow
• Is there any efficient non-linear note-taking software?
• Software/App for electronic research notebooks

Answer 2

I cannot say that this is the perfect way to do it, but I'll cover how I do it.

Background on my perspective: I'm a mathematical/theoretical epidemiologist who came up not through applied math or one of the related fields, but through a circuitous route from laboratory biology. So, as far as I can remember, Science Is Done In Lab Notebooks.

A couple work patterns I've developed:

1. My lab notebooks are actually notebooks. I experimented with a Wiki for awhile, and just using something like EverNote, but I found they didn't work for me. They didn't capture the same feeling of permanent documentation, and freedom of notation. So now I use notebooks from Black and Red. Mainly because the generic black-and-white composition books remind me too much of undergrad.
2. Like wet-lab science, I keep one notebook per topic - but to address your problems, I tend to broaden the topics. A particular focused project (like something I'm being hired to consult for, or a one-year RA, etc.) gets its own notebook. Projects involving the modeling of a particular type of disease all get a notebook. Side projects involving a different aspect of theory...you guessed it, get a notebook. I've also got a small notebook that is literally "random musings and paper ideas". So rather than one lab notebook, I have several. These are identified in ways that are sometimes logical (a GitHub sticker on the very code-heavy project book) and governed by whimsey.
3. Always have the book out. Just like a wet-lab notebook, if you're working on a project, you should have the book out. Don't be afraid to write random musings. If you've done some math in your head, write it down. See also diagrams and sketches. Print stuff out and tape it in the notebook, just like one might do pictures of a gel or western blot. In the notebook currently on my desk, I've got several graphs, and even a printout of a question over at CrossValidated that makes up some portion of my thoughts at that point.
4. Reference yourself. Occasionally, as you've noted, ideas cross-polinate. I page number all my lab notebooks, so occasionally I will end up referencing myself. "See Lab Notebook X, Page 47."

Answer 3

First, the fact that you're asking the question is already a good step forward. Being a researcher is about managing ideas and data, and some researchers don't fully recognize it!

---

After a few years as a researcher in theoretical chemistry (all computer, no wet lab), I have seen pretty much every kind of notebook possible:

• Single notebook documenting your progression linearly. I started with that, it hurt a lot.
• No notebook, everything in files: put a 00README file in each topic/subtopic directory, then simply write in there to document your progress. In meetings, either type directly (if you have a mobile device) or write meeting notes then scan them.
• Thematic notebooks (well described in other answers).

What you need is something that works for you. Draw inspiration from the examples above, the other answers, and try it! If at first you don't succeed, change the way you do, identify things that prevent you from fully using the workflow you set out to.

In particular, important factors to be considered are: how much do you travel? where do you work from? For me, thematic notebooks are a no-go because I work from the office, work from home, work from conferences, work from hotels and planes, work from vacation (I know, I know…). Thus, I can't carry around 5 or 10 notebooks.

To be honest, the combination I have seen used very efficiently (by me or people around me) is:

One single “take everywhere” notebook (I make it a Moleskine: because it feels special, it makes me want to write in it). Each lasts about a year. They are organized as follow:

• at the back, few pages reserved for key information (important dates we set, strategic research decision, contact numbers for project leads, that very funny quote that had me laughing throughout an entire hour)

• at the back, just before that: a few pages per project, with very high-level tracking of the progress (I write about one line or two per week, when the project is running). This allows me to remember

• from the front: everything else, chronologically but clearly dated and labeled. Notes during meetings, notes about important stuff running and its analysis when it has finished running, etc.

• everything else (long math derivations, paragraphs of text I started to write, full meeting minutes, detailed graphs): electronically

You can further refine: a colleague of mine uses small colored sticky index-tabs on the top of the notebook to index the content that turns out to be the most important (important results on each project, color-coded). I cannot force myself to do that, but I must admit that it really helps her access older information quickly.

Answer 4

This is a problem I struggled with for three years. My background is computational physics, and like you I have many active projects at any one time. My work involves three main activities: scientific research, software development and paper writing.

During my first post-doc position of three years duration, I managed to come up with many different ideas for research and features for the software package I was developing, but simply lacked the time to progress with them all. I flitted from one to another, trying to devote a little bit of time to each idea, but never making real progress on most. I also never kept a proper lab book, just having a directory on my computer for each project. This meant that I often left work untouched for weeks, and couldn't remember what I did last, often having to rely on time stamps to see which data was newest - very unsatisfactory indeed.

However, I recently made two changes which have made a massive difference to my ability to effectively deal with many projects. The first is dedicating large blocks of time (3 hours generally) to a single project and focussing on that one task exclusively - no email, no meetings, no Internet if I can avoid it. The other change is setting out a master plan for each month. I prioritise the best quality ideas and time critical work such as project reports, conference abstracts and talks. Other projects are put off until I have free time or they develop into something better, at which point it becomes a priority anyway.

My tool of choice from a practical perspective is Evernote. The ability to group collections of notes and restructure/combine them as needed is crucial to dealing with many related ideas which may converge or diverge at different points in time. Also when I finish a block of work on a project I religiously add my thoughts and results to the lab book, so that next time I come back to it (which potentially could be weeks or months hence), I know exactly where I got to. I also refer to specific directories on my computer so that I can always recreate the data/figures at a later date, without having to hunt for the right directory.

When I have a new idea I start a new notebook under Research, which contains a few bullet points about the idea - enough that I can recreate the thought should I completely forget about it (it happens!). Later I will usually come back and add a project plan, from which point things are usually pretty linear. Given each idea/project has its own directory of notes, I always have an overview of what is going on, while the linear series of notes catalogues the thought processes and data. Using this approach I find it much easier to keep control of my research and its direction, and I am much happier and productive for it.

For me an electronic lab book is essential. As others have mentioned if you travel a lot then physical notebooks are suboptimal. I still have a book for quick notes and equations, but they get typed into the lab book/LateX as soon as I get to the computer. Although my lab book is currently quite small, as it grows I want to be able to search for data and text. There is of course a potential downside with my current solution - it relies exclusively on the existence of Evernote. If the company fails for some reason then I potentially lose all my notes (since they are hosted on their servers). I am always on the lookout for new software which does the same thing, and preferably open source which takes away the heavy reliance on a single commercial entity.

Answer 5

I few years ago I moved to MacOS and discovered a journaling program called Day One. My journal is automatically backed up to a server. You can have one journal per topic, or use tags to identify research projects within a single "research" journal. Many of my thoughts apply to more than one project, so I use tags and one journal.

Perhaps the most important feature is that there are apps for iPhone and iPad, and you can add pictures to the journal. So, when I am talking to someone and she writes something on the board, I can take a picture and add it to a journal entry. I can also take a picture of a calculation I do on paper, or export a mind map (I love Inspiration mind-mapping software) to an image and import that into the journal entry.

I also have great ideas while jogging in the park, at the gym, driving to work, and I can add them to my journal from my phone. I can add longer notes using my iPad when at a conference.

Since it is journaling software, everything is organized in date order, but you can click on a tag to see just the entries with that tag. I don't know how I lived without it.

You can export your journal to pdf for extra backup or sharing.

There might be something for Windows too (not OneNote, which is more like a Wiki). I had problems with EverNote losing my notes, never to be found again.

Answer 6

If you lab notebook could be used to document a patent-able device or process then you must use a stitched notebook with numbered pages, and do not remove the pages.

Even under the first-to-file rules, some cases can draw on the history of notes and creative documentation. (e.g. same claims on the same day)

As annoying as this is for modern methods (hyperlinked docs, etc), there is no alternative yet.

(Source, a 20yr patent attorney's career)