How to read papers without falling into a rabbit hole?

Question

Often while reading a paper, I will see a citation that that seems more relevant than my current one. I'll switch to the new paper, only to experience a similar feeling. Or maybe I need to look something up on Wikipedia. An hour later, I'll still feel as if I didn't really digest anything. I will have skimmed 2-4 papers' introductions and conclusions, scratched some notes on the side, and generally feel overwhelmed.

Is this a bad habit? An abnormal feeling? How have others dealt with this to become more productive readers?

I'm fairly new to research, if that matters.

Answer 1

Your problem is quite common among researchers. Actually, it's not really a problem, being overwhelmed like this is just natural. I and all my friends and colleagues face it.

How I overcame this issue: I try to focus on one paper at a time. Try starting reading the latest research paper on a particular subject and go back chronologically. Print the research paper in hard copy, leave your computer and cell phone (if possible) behind and start reading the paper. The point here is to avoid internet access. No matter what question you have, do not search for it right away. Write it on the side notes and keep reading. Most of the time the answer will be in the later sections of the same article. Even if you do not get the answers, once you finish reading the paper, you can go online and find all the answers. That's how I read literature without falling into a rabbit hole.

Also, reading the Abstract first, then the conclusion, and then the rest of the paper is also effective.

For me, changing place of study also helps to focus and good instrumental music is always a plus.

Answer 2

(For context, I am in mathematics, and worked on it for almost 20 years before the convenience of the internet...) As suggested by @JonCuster's comment, I think this is what is supposed to happen when one is studying (!). That is, I think genuine study of the literature is "going down the rabbit hole"... and/but not giving up or bailing out somehow. Sure, sometimes there's a "grass is greener on the other side" feeling, but when looking at the research literature it is entirely reasonable to be stubborn in the sense of insisting on explanations.

It struck me that you mention "after an hour, and looking at 2-4 papers"... when I often find myself looking at papers all day long with the conclusion being that there were things that I didn't understand, or wasn't aware of, or had forgotten, that have surprising/significant impacts on the issue I'm considering.

It is true that this (I claim genuine) seeming-inefficiency is wildly incompatible with the assembly-line notion of "research" that we find ourselves being pushed into for various reasons. Hmm... :)

Making notes with good bibliographic indicators, and typing-up these notes rather than leaving them handwritten, and dating everything clearly, is the only thing I know to keep back the deluge. Dating, especially, so that months or years later you can see at least your own personal chronology of awareness is very useful, I think, since it can explain things to your future self about your present and past self's behavior.

In a similar vein, rather than attempting to organize computer files "by concept", in some cases (when the concept is not entirely clear!) it is simply best to organize by chronology.

Answer 3

I have previously written an answer to the math version of the same question. I see no reason why it can't be applied to this case as well.

The human tendency is to prefer worrying about the most recently raised concern: You are reading paper X. You see a term T you don't recognize. You decide to pause reading X to quickly look up T (of course, underestimating how long that will take). You find a paper Y explaining T. While reading Y you find another unclear term U. U has been encountered just now, but T and X were minutes or hours ago - of course U seems like the more urgent thing to figure out. Next thing you know, you're in a meeting about X, you don't know a damn thing about X, but you sure learned everything there is to know about U!

So the human mind, at least in my experience, operates as a stack (in fact a leaky stack, since our attention is quite limited and things at the bottom often become forgotten instead of just delayed). The stack happens to be the defining feature of depth-first search algorithms.

By many relevant metrics, depth-first search (DFS) is a particularly bad choice for reading papers. For instance, presumably the paper you are reading is expected to contain the most important information to you at the time; if there was one that seemed more important, you would read that instead. But with DFS, you will spend a lot of time reading other papers, only distantly related to the original paper - ie. you will waste your time on less useful things.

Because human knowledge is vast, your attention will become exhausted long before DFS hits the wall and starts returning to the original topic.

Probably the biggest reason more efficient approaches (for instance breadth-first search [BFS]) are not more common is that they require additional hardware. Namely, a pen and paper to write a list of things to look up as you read the paper, so that you don't have to split your attention between reading the paper and remembering this growing to-do list. One also has to fight one's own laziness when going to grab a pen and paper, and then (gasp) actually writing, which is much slower than thinking.

Also, the only important point isn't the order in which you look things up. There is also a pruning issue. Many people (including myself) will overestimate how important a term is to understanding the main point (quite a silly habit, since it's essentially trying to guess what it will take to understand a text you have yet neither read nor understood). With a BFS style approach, often it turns out that most of those things you thought you should look up don't really matter and you don't need to look them up. With the DFS, it is much harder to tell what terms actually matter, which ones are irrelevant and which ones become obvious by the end of the text.

Basically, you have to exercise your patience and rebuke your inner sloth. Finish reading your current thing first before looking everything up. Don't worry, you won't forget - just write down what needs to be looked up, with references to where it occurred if you are very worried. Yes, you do have to physically write things, which is clumsy (hence why I say rebuke your inner sloth) but absolutely necessary to overcome some crucial limitations of the human brain. You can type instead of writing, or highlight, or draw !'s on the margins - doesn't really matter, so long as you use a physical (as opposed to mental) means of recording it, and finish what you're reading before starting to read other things.

Answer 4

It is interesting that you used the concept of "rabbit hole" when you are new to research. A couple of things:

Uncertainty is Always There: You read it right. The "rabbit hole" is always there. In fact, at first, it is the "black hole" that sucks you in and you are amazed/sad/angry all at once!. This is normal because the area is new, you have no idea how to make something out of it. As you progress the hole starts to fill in more and more, however, the uncertainty is always there but your approach to the research becomes more and more clear and becoming more smart and you will guess what your contribution will be. This brings me to the second point.

Contribution: Take your area, and work toward a contribution. You can never be 100% sure about a contribution until you can prove it in a respectable publication. This is the whole point, you don't fill the "hole" completely, however, you fill it a little so the next researcher that comes to your area, feels a little more comfortable in the "rabbit hole".

Read Read Read!: Keep reading, take notes, see who is doing what, and then after a year or two, if I ask you about your area, you know the history and you know who did what. Most of the time it comes down to a few people. You will be amazed at how many few people are bringing your research area forward.

Answer 5

Getting hold of a good literature survey paper helps especially if written recently...given that it exists for your topic.If so you probably wont do better than citing the relevant paper mentioned in the survey. Hand-books also help identify which papers ought to be the go to papers for citation.

Answer 6

One way would be to write down every term or phrase that spark your interest and look them up later.

The best way in this day and age, in my opinion, is to open up each paper you plan to read in a new tab in your browser and sort them in order of importance. For example, the paper that you are currently reading should be in the active tab; the next tab should be the paper you want to read after you stop reading your current paper, and so on. Then you can easily stop reading one paper, switch to another and deprioritize what you were just reading by moving it down the tabs list so you can get back to it right away or a bit later. You can also utilize favorites to add papers you want to go back to later.

However...

Falling down one of those rabbit holes led me to changing fields and a different career path. I am very happy about that and I think others might be too if they just follow their thirst for knowledge and do not resist it. It is very natural for a scientist (and for any person!) to be open minded and constantly search for something more interesting, fascinating, amazing, and readjust academic and personal goals accordingly. Whenever you find something interesting that seems more important than what you are doing now, check it out, maybe it is! If it's not, drop it like a rock and go to the next paper, or back to your previous one. You can get really, REALLY fast at multitasking like that using browser tabs/favorites to organize/reorganize your reading list. Use CTRL+F searches to quickly find the phrases you need. Modern PDF viewers also utilize tab systems, which gives you even more organizing power. Get good at this and those deep rabbit holes won't seem that deep anymore and you will find plenty of wonderlands down there with much less effort.

Answer 7

As Mirza Awais Ahmad already said, if you know that there is one paper you need to focus on, a good solution is to print out the paper, turn your phone off, leave the office, and go read it in a cafe/park/at home/on holiday/in the library without the ability to download another paper. Bring a notebook with you and make notes on paper or in the margins. If there's a reference that seems important, just make a note of it for later, when you're back at your computer. Setting aside time like this can be a greatly enjoyable experience, especially if your normal mode is skimming.

If the issue is that you don't know which papers you need to focus on, an excellent solution is to join a reading group. Or, if there isn't an appropriate one in existence, start one yourself. Find some likeminded people with an interest in a similar area, decide collectively which paper to read next, and set a date to discuss it. This not only provides a great motivation for sticking to the target paper, but you also get the help of other people in deciding what's important, and the subsequent discussion can be an enormous help in understanding both technical details and the broader context. You'll almost certainly be doing your colleagues a favour as well by starting a reading group, and it will also give you experience organising groups of people, which is invaluable for a future research career.

Answer 8

Some tips for dealing with the rabbit-hole when starting in a new area:

• Try to find an existing literature review of the field: When you are entering a new field, the best resource to start with (if it exists!) is a well-written literature review that takes you through the history of the field, and tells you about the main papers, research questions, results, etc. For example, Fang and Moro (2010) gives an excellent summary of economic literature on discrimination, which is pitched with a level of detail that allows a good overview of models, without wading into every bit of minutia. This kind of resource allows you to learn about some of the main papers in a field without a huge reading investment.

• Find and read the "core" papers in the field: After you have read a good literature review of the field, the next step is to read the "core" papers in detail, to get a good understanding of the main investigations and results. (Even if there is no literature summary to find these, when you look up papers in a new field, most of them will cite some of the core papers in the literature somewhere in the introduction, to establish themselves within a context of broader work. If you see that multiple papers in the field all cite certain core papers in their introduction, this can be a good indication that those papers are important starting points in the literature.) Reading and re-reading the main papers in the literature is a good way to get good core knowledge of the main results of the field. This is a good platform to learn the field in the long-run.

• Now you're ready to go back down the rabbit-hole: Once you have a good overview of the field, and have read some of the core papers, you are in a good position to go back down the rabbit-hole of reading individual papers, which give you leads to more papers, which give you leads to more papers, etc. Welcome to academic research!