48

This may be a weird question, but I often wonder why the default way of writing about a given methodology is describing the method as if it "fell out of the blue". Admittedly, not all research work is written like this and perhaps this is not an issue for very experienced researchers/practitioners, but I believe it would be much more insightful if authors wrote the steps that led to the specific decisions when designing a new method. So, instead of describing a new method, the author would write the problem and motivation (this is usually done), and then elaborate on the thought process that led to the final design choices. I don't mean hand-holding on basic stuff, but to describe things like "we wondered if there was a way to solve X. Y is a popular method to achieve Z, a property required when solving X for the reason K. However, Y needs to be tweaked in the following manner so that it also asymptotically satisfies W" or "we tried this way of solving it, but it did not work due the following reasons [insert reasons], so we decided to try this instead". Was a certain property coincidentally satisfied by your decision choice, or did you reverse engineer a method that satisfies that property?

This may seem like a childish idea and most people probably don't have the time or interest in such a writing style (or may even feel insulted by overly detailed descriptions). However, I believe it would be a much more valuable contribution to science because:

  1. Researchers would learn from each other different ways of thinking about a research problem and the strategies to solve it
  2. Researchers interested in building on the presented work would know which ideas did not work (preventing them to pursue dead-ends), and if they know better ways of solving a particular sub-problem they could easily improve the method.

What are your thoughts on this?

17
  • 9
    This seems entirely opinion based. But researchers aren't cut off from that "process" stuff. It just appears elsewhere than in the papers themselves. Moreover much of it is irrelevant to the results and could even be misleading.
    – Buffy
    Jul 28 at 12:57
  • 7
    With exceptions for reviews and the like, journals typically have page limits, e.g., eight pages. Even with online supplements available, conciseness helps readers keep their heads above the rising tide of publications. Maybe.
    – Ed V
    Jul 28 at 13:04
  • 7
    I've heard of honors Calculus classes that do teach this way, but are very confusing for the student. Also, looks at books that do. For example, a Radical Approach to Real Analysis does describe the history. Last, the history of science field often covers the how science discoveries occurred and the thought process. Jul 28 at 13:28
  • 5
    There's been a lot of prior discussion of this question in the scientific literature. See responses to Peter Medawar's Is the scientific paper a fraud?, for example. I also recall a similar sentiment expressed independently. Jul 28 at 14:16
  • 3
    @Schmuddi Good to know! One nice thing about this particular stack exchange is learning just how varied academia is.
    – Ed V
    Jul 29 at 10:10

11 Answers 11

44

Thought processes are messy.

If you wrote out the thought process behind the typical papers I contribute to, you'd have to distill hours of weekly meetings, circular avenues where the same idea comes up 2 or 3 or 2 dozen different times before it gets incorporated definitively, endless iterations of experimental design, dead ends and failed experiments where everything starts over, side projects that unexpectedly inform a central one, shower thoughts, and pub inspiration.

In a good writer's hand maybe it could turn into a novel, but it wouldn't be the most direct way to explain the main findings of a work and contextualize those results within the literature. As Richard Erickson comments, history of science books often do get into the "process" behind discovery and they can be a great read (Ruse's The Darwinian Revolution is a personal favorite), but it's not reasonable to expect every scientist to go into this level of detail for every paper nor reasonable to expect their audience to invest the time to read it.

I do see "thought process" content in many papers, but at that point it is usually made into a much simpler, linear story to avoid all the meandering.

13
  • 17
    The "thought process" behind some discoveries went on for a hundred years.
    – Buffy
    Jul 28 at 13:55
  • 1
    I agree. I am not suggesting adding every single step but rather the core, trimmed down ideas. Jul 28 at 13:57
  • 1
    @user3653908 Maybe your field is different, then; I see the core/trimmed down thought process often, usually you'll find it in the introduction section. From a textbook perspective I would also say that biology textbooks I've experienced are usually written with history/thought process in mind, at least in part. They usually use a few key experiments to demonstrate "how we know what we know" and familiarize students with scientific methodology rather than just drilling facts. They don't go into everything like this, though, for the same reasons I mention in the answer.
    – Bryan Krause
    Jul 28 at 14:43
  • 17
    @user3653908 My cynical view of a lot of papers in the ML field is that there often isn't much thought process, unfortunately. But I am definitely biased in thinking mostly of papers that overlap with my own field where the papers are less "new method in ML" and more "I know sklearn exists, what if I use some of the methods there on this brain data. Brains are cool right?" But less cynically, yes, length limits are surely going to cut down on prose when there is other meat to share. Hopefully they at least cite similar approaches; if not that's just sloppy rather than a style choice.
    – Bryan Krause
    Jul 28 at 14:51
  • 1
    Even though it's a bit messier, I find it more fair, relatable and educational to read "I did not know, so I asked on SE and Bryan Krause told me that [..]" instead of just stating the final result and including a link in the references.
    – Džuris
    Jul 29 at 20:34
20

My answer is from a mathematics perspective.

I like the analogy of exploring an alien landscape for doing research in mathematics. In this analogy, the ideal mathematical paper reports on having found some astonishing landmark together with useful instructions of how to get there.

Good instructions for how to find a place will look very, very different than the journal of the first explorer to get there. In the latter, you'd have stuff like

I was trying to reach the summit of that mountain, but at some point I found myself separated from the summit by a deep gorge. But when I looked around, I spotted that serene lake between the tree tops. So I climbed back down and tried to make my way to the lake. Then I caught malaria and walked circles for a while hallucinating. After recovering, I made a lucky guess and stumbled upon the lake again, and it really was very beautiful.

But when we are actually dealing with math research, it probably makes much less sense. Reaching the point where you understand an idea well enough to even tell a fellow researcher about it can be hard work. There is "For a few months I thought about whether X could be a good way to attack my Y problem. I don't remember why I ever believed that this might have worked out". There is "one morning I woke up, and under the shower it suddenly all made sense". None of these are particularly helpful for a reader.

9
  • 3
    Waking up with an insight is pretty common, actually. The brain works even at rest. Sometimes even better at rest. Mathematical "dreams", on the other hand, often have very strange logic.
    – Buffy
    Jul 28 at 15:03
  • 9
    @Buffy I think Arno's point is not that it's uncommon, but that it's irrelevant or at least unhelpful to a reader trying to understand the paper to know some step was a result of an insightful sleep. Well, maybe it could encourage the reader to develop better sleeping habits.
    – Bryan Krause
    Jul 28 at 15:09
  • 2
    @BryanKrause, actually, I was "catapulting" the writers idea, not criticizing it. But yes, sleep and exercise are good research habits.
    – Buffy
    Jul 28 at 15:11
  • 5
    @user3653908 My point is that you seem to vastly underestimating how messy the proof process often is. I see explanations of the kind you wish for often enough that it is very plausible to me that most mathematicians include them whenever they had a clear rationale for trying something that they can express easily in words.
    – Arno
    Jul 28 at 22:05
  • 4
    @user3653908 I still think you're underestimating the scale of the question, here. I've written <10 page papers for which the those sections would stretch into hundreds of pages. Jul 29 at 13:55
12

In papers from my field (chemistry/materials science), it is actually quite common to do exactly what you suggest. However, you will not find anything of that in the typical methods/experimental sections because those are reserved for a description of what was done. The why is part of the discussion part of papers where authors explain what the underlying reasons are for their results, but also should justify their methodology if it is not obvious. Sometimes, this involves giving short mentions of attempts that failed to give the desired outcome.

10

I can speak to why this is the case in mathematics; I suspect this bled over into the physical sciences as well but that's just a hunch.

When I was an undergrad I tried to read Rudin's Principles of Mathematical Analysis and at first I was totally perplexed -- seemingly out of the blue, he'd pull out the value of what some constant had to be in order to make the theorem go. I felt very stupid and asked my dad about this. He explained to me that this style of exposition is deeply ingrained in the culture of mathematics, going at least back to Carl Friedrich Gauss, who famously said, "A good building should not show its scaffolding when completed." He then gave me some strategies for reading, understanding, and writing proofs, and part of that involves reading and writing in an order other than the order in which it will be presented. None of my professors gave me or anyone this advice, we were just expected to figure it out or fail. Looking back, I think this was basically a form of gatekeeping.

There have been mathematical works that are presented in a different style, with much more background and exposition. I think one of the most notable examples is Melissa E O'Neil's work on random number generation. The PCG RNG paper did not appear in a peer-reviewed journal for some time because the ones she sent it to objected to the style of her writing, which presents much more of her thought process than would be customary for a math paper. You can read more about the history of the PCG paper from O'Neill herself here. The algorithm has since been included in several major software packages including numpy. Personally, I find the PCG paper to be a great read, and I think the fact that she had such a hard time publishing it is a damning indictment of the culture of mathematics.

So, to summarize, the reason more papers aren't written in this style is because (1) it's an ingrained habit that comes down to the present day through Gauss and other early 19th century mathematicians, and now (2) the consequences of trying to change the culture around this include doing a lot of unrewarding work and getting rejected repeatedly.

3
  • 1
    This answers my question. I was hoping to start a good debate on how things could be better, but it seems most are focusing too much on how involved the process is rather than how something would be vastly better than parachute definitions and the likes. Hopefully, this discussion will motivate more people to slowly abandon this terse style. Jul 30 at 15:24
  • Robert Ghrist's book Elementary Applied Topology is an interesting example that very much breaks away from the traditional writing style, but that was self-published. A lot of people have also taken to blogging for the things that can't quite go in a refereed publication but which are still worth writing about. So it seems that the route is to seek other venues rather than to change the way traditional academic publishing works.
    – korrok
    Jul 30 at 15:45
  • @user3653908: " Hopefully, this discussion will motivate more people to slowly abandon this terse style." It certainly will. Look at the number of upvotes the question received.
    – Tho Re
    Aug 1 at 21:25
7

I am not sure that we really have access to our actual thought processes behind our research, just the post-hoc internal explanation that we remember. It would require us to actively analyse and record our thought processes as we go along and who has the time for that. This may just be me, but if I am working on something my ideas and assumptions change and evolve as I perform experiments (partly because the next experiment are shaped by the results of the previous ones). I don't think I am able to roll back to the version of me at the start of the project and actually evaluate what I thought at the time. I think it is a mild case of the incommensurability of Thomas Kuhn's paradigms (but on a smaller scale).

Generally when I write a paper, I try to create a logical progression of ideas to explain what I found out during the course of the study. This is very rarely the same as the historical progression of ideas, with all of its blind alleys and "my brain hurts" moments (again that may be just me).

We are all prone to post-hoc rationalisation of events, it is human nature. I strongly recommend the film Rashomon to anybody that hasn't already seen it. I doubt the Rashomon effect named after it only applies to recollection of external events, or to situations involving personal gain/loss. I am not convinced that we are really objectively that aware of our thought processes and motivations.

1
  • 1
    Interesting points. Thanks.
    – Buffy
    Jul 29 at 14:34
6

When you read a fiction book, you never have the author's thought process on how he designed the book. You may have the narrator's thought process in some books, but not the author. You only get the final output.

When you watch a movie, you don't have the thought process of the director included in the movie. You only get the final output. That would be dump in many cases, as it would distract from the movie.

A research paper is this, the final output of a research process. A finely crafted piece of science whose goal is to convey information to fellow researchers in the most efficient and concise way.

If you want an example of reading about the thought process of some researcher, you can read Birth of a theorem by the mathematician and Field medalist Cedric Villani.

2

(I’m coming mainly from mathematics, with some crossover experience in theoretical CS.)

Most good academic writing does include a bit of “thought-process” explanation. The style you describe as “the default way” — writing as though everything “fell out of the blue” — does exist, but it’s not the default in fields I know (though it was more common a generation ago, in some areas of pure maths). Papers with no motivating explanation at all are seen as unpleasantly dry; it’s not uncommon for referees to ask for better motivating exposition.

But it’s usually just a little: Too much “thought-process” explanation is not as helpful to readers as you seem to expect. I’ve read papers that tried hard to explain a deep and subtle thought-process, but came across just as impenetrable waffling, and left me wishing the authors had just stuck to the facts. The main problem with such attempts is described well by Brent Yorgey: Abstraction, intuition, and the “monad tutorial fallacy”

So in sum, good authors are certainly conscious of this aspect, and usually choose to include some thought-process explanation, but not much. If you think more would be better, then be the change you wish to see in your field, and use a bit more in your own writing. But be aware of the reasons why most writers use only a little — remember the parable of Chesterton’s fence!

1
  • 1
    +1 Chesterton's fence is very apposite! Very relevant to debugging computer programs, but I suspect my students may appreciate the reference as much as I do! ;o) Jul 31 at 18:27
2

For many research-level results I have obtained, the thought processes involved hundreds of wrong turns, including useless definitions and wrong proofs, and it would be really silly to include all those. Unfortunately, it is rare to be able to cogently deduce the results without significant trial and error, otherwise it would not be worthy of research, would it?

However, it is true that in many cases one can come up with a fictitious thought process obtained by splicing together the actually useful ideas that led to the final solution. And that ought to be included in any good exposition of the results. Sadly, there are still page limits for many peer-reviewed publication venues, putting a strain on the desire to give a verbose story of how to come up with the results.

1

Also from a mathematics perspective.

I was working on a problem for two years now, and, recently, in the process discovered another (in my opinion) nice connection/result. I wrote a paper about that (under review). I also felt it might be helpful for a potential reader, to better evaluate and understand the results, to give away some history and thought processes involved. However, from a logical point of view, it seemed more natural to me to present it the other way around: first present the discovered result and after that the application to the original problem that brought me there. Nevertheless, I gave a "personal story" in the introduction, I wrote something like this:

Now, maybe it is best if I give a little bit of a personal story about the results presented here, and that they are, in some sense, the result of two strands of thought.

[... following approximately one page of text, describing a collaboration, where the original problem came from and how it led to the present results...]

As is most often the case, and to have a clean separation between a more group-theoretical part and a more automata-theoretical part, the presentation does not follow the order of discovery in this sense.

After that, I continued to present the results. I hope the reviewers like my "historical account" ;)

1

Different people follow extremely different thought processes. Thus, it is not, in general, extremely productive to force all people to arrive at a certain conclusion in the same manner. A closely related concept is described quite well by physicist Richard Feynman in this video on youtube(section beginning at 55:01) (from a BBC special "Fun to Imagine").

Consequently, presenting with a focus upon the important conclusion is a strong strategy. If it is necessary, one can subsequently build "walls" around the core idea being presented by describing other attempts to solve the problem that were eventually unsuccessful.

1

Let me give a different perspective, coming from mathematics. Other answers have already explained the reasons why papers generally don't have so much thought process, so I won't rehash them here. Instead, let me contend that giving the full thought process has its own merits, and that more examples of this would be a positive thing.

The most prominent example I know of where the full thought process is given is in Grothendieck's Pursuing Stacks. It is hundreds and hundreds of pages long with endless digressions. Nevertheless, it has been influential, and even personally, it has taught me quite a lot per page, though I don't work in this field. In fact, it is by far the most enjoyable mathematical text I have ever tried to read.

Why do I think this is? Briefly, because I think it models discussing math with others much better than textbooks/papers. Generally speaking, discussing math with others is important for learning because it allows for the freedom of exploring ideas with another mind. Textbooks/papers are important because they give you shortest path to various goals and prove a lot of important results, which you can study on your own. So I see two needs which papers giving the whole thought process may fill:

  1. For people who don't have people they regularly discuss a certain area of math with.
  2. For people looking to just enjoy math, not concerned about publishing papers in a certain field.

Finally, let me add that even from a "practical" standpoint, texts like these may have the potential (apart from the actual ideas they may contain) to expose students to the unconstrained nature of doing research, which at least for certain students may be important to building their own tastes.

Not the answer you're looking for? Browse other questions tagged or ask your own question.