Why should a publication present results in a way that significantly differs from the discovery of said results?

Question

In the various projects I've worked on, professors have insisted that a writeup for a journal submission should state and prove only the final results (rather than describe the process of arriving at those results). I see how this makes things concise, but it seems to obscure the process of discovery.

Here's an example of this situation from a mathematics paper I've worked on.

What I did:

1. I have a direct proof for "X".
2. After looking at my direct proof for "X", I see that I could generalize "X" if I prove "Y" (it was not immediately clear that proving "Y" was useful for proving "X").
3. I prove "Y", then prove "Generalized X" (duplicating some work from the proof of "X").

What the paper does:

1. Prove "Y".
2. Use "Y" to prove "Generalized X".
3. Refine the proof of "Generalized X" ever so slightly to get "X".

My frustration:

I feel like the paper is good reference material, but it's not educational. I think a reader would stand to benefit much more from a direct proof of "X", since it shows why I would try to show "Y" in the first place.

Answer 1

I think this comes down to the subtle but important difference between communication and education. The target audience of a research paper is experts in the field, not students. Most readers want to know what your theorem says, why it's important and relevant, intuition about why it "should" be true and why the proof works, and (sometimes) the details of the proof. Few people are reading in order to learn how to prove theorems, which is what your suggested presentation would serve.

Having said that, it's possible to take the "uneducational" approach too far and actively cover one's tracks, so that the paper is hard to understand but looks impressive. By all means, if an informal discussion of the special case "X" will help give intuition about the main theorem, include such in your introduction.

Answer 2

What will be most useful to readers, for understanding the final result?

The primary purpose of a research paper is communicating its original results. Educating the reader about how research works in general is great, but should generally be considered a secondary priority in a research article; if you want to focus on that, you can write it up separately in a different venue.

On the other hand, sometimes the final result — if simply given on its own — may seem unmotivated, or overly abstract, and will be difficult for readers to understand. In that case, it can be very helpful to lead up to it by presenting some edited highlights of the process of discovery. (Or by giving a worked example, or a high-level overview, or various other things.)

So if your co-authors are really saying “You should always give just the final result!” then they’re being shortsighted and unreasonable. But if — and I guess this is more likely — they’re suggesting it because they think it’s clearer for readers, then it’s a judgement call about exposition and pedagogy, and you need to discuss it as such.

There’s not much value in recording the process of discovery for its own sake. There is great value in describing the process of discovery if it will give readers a path to understanding results which otherwise might seem abstract and impenetrable.