Let me mention some points from two articles on this topic, with a request for more such sources:
* Steven Pinker, _Why Academic Writing Stinks_, Chronicle of Higher Education, September 26, 2014. ([Paywalled link](http://www.chronicle.com/article/Why-Academics-Writing-Stinks/148989/), [download link](http://www.chronicle.com/article/Why-Academics-Stink-at/149105).)
* George Gopen and Judith Swan, [_The Science of Scientific Writing_](https://www.americanscientist.org/issues/pub/the-science-of-scientific-writing/99999), _American Scientist_, November–December 1990.
---
In the first article, Pinker considers three explanations commonly given, and gives three of his own.
The cynical **Explanation 1** is that there are certain fields, and certain academics, in whose case the bad writing is *intentional*: to bamboozle the readers, or as "emperor's clothing". (For likely examples, see Dennis Dutton's [_The Bad Writing Contest_: 1996–1998](http://www.denisdutton.com/bad_writing.htm) and [_Language Crimes: A Lesson in How Not to Write, Courtesy of the Professoriate_](http://www.denisdutton.com/language_crimes.htm).) Even in the *unfathomable* example given here, this is a possibility. But having recognized this, we can move to the more interesting cases, where the obscurity is unintentional.
Then Pinker considers **Explanation 2**, academics’ “self-serving” one: that there is inherent complexity in what is being discussed, and therefore difficult writing is unavoidable. Although this is true to some extent (every field has its jargon), it is not a complete explanation. Even expert readers are baffled by articles from their own sub-sub-subfield. Pinker gives this example:
> Participants read assertions whose veracity was either affirmed or denied by the subsequent presentation of an assessment word.
which he considers "not as concise, accurate, or scientific" as the plain:
> Participants read sentences, each followed by the word *true* or *false*.
He also mentions an **Explanation 3**: that gatekeepers of journals expect difficult language. This is not true: every field has some papers that are well-written.
Instead, Pinker comes up with three other explanations (which I reproduce here in reverse order).
**Explanation 4** is that academics simply have few (or no) incentives for writing well. Clarity requires practice, showing drafts to readers, and a toolbox of skills. This is a lot of work and the profession does not reward it (directly, at least). (And there are even some incentives *against* writing well, along the lines of the recent article on [perverse incentives in academia](http://online.liebertpub.com/doi/full/10.1089/ees.2016.0223).)
**Explanation 5** is the Curse of Knowledge: once you know something, it becomes hard for you to imagine what it is like for someone else to not know it. (Related terms: “false consensus”, “illusory transparency”, etc.)
For example, jargon and abbreviations that you personally use frequently can become second-nature to you while remaining opaque to readers (even your colleagues). There are names in cognitive psychology for two phenomena that explain *why* authors fail to think like their readers: “chunking” and “functional fixity” (jargon, but he explains them).
> The amount of abstraction a writer can get away with depends on the expertise of his readership. But divining the chunks that have been mastered by a typical reader requires a gift of clairvoyance with which few of us are blessed.
> […]
> They are not trying to bamboozle their readers; it’s just the way they think. The specialists are no longer thinking—and thus no longer writing—about tangible objects, and instead are referring to them by the role those objects play in their daily travails.
> A psychologist calls the labels *true* and *false* "assessment words" because that’s why he put them there—so that the participants in the experiment could assess whether it applied to the preceding sentence. Unfortunately, he left it up to us to figure out what an "assessment word" is.
The other explanation from this article (the first and longest one given by Pinker) is based on a theory of different styles of writing, in terms of what kind of conversation authors imagine themselves to be having with their readers. Instead of writing in "classic style" (writing as presentation, for directing the reader's gaze towards the truth, successful when clear and simple), academic writing is often in a blend of two styles: a "practical style" (writing in a fixed template, for satisfying a reader's need for a particular kind of information) and "self-conscious style". This is **Explanation 6**, but it needs a bit of elaboration on what this "self-conscious style" is, and how it manifests.
In self-conscious style, "the writer's chief, if unstated, concern is to escape being convicted of philosophical naïveté about his own enterprise".
> Their goal is not so much communication as self-presentation—an overriding defensiveness against any impression that they may be slacker than their peers in hewing to the norms of the guild.
Pinker considers many common flaws in academic writing to be symptoms of this style: *metadiscourse* (writing about the writing itself, as in "The preceding paragraph demonstrated…" which could be "As we have seen…"), *professional narcissism* (talking about the world of their profession, rather than about the world of the things they're studying: "In recent years, an increasing number of psychologists and linguists have…"), *apologizing* (about how complicated/controversial their topic is), *shudder quotes*, *hedging*, and *metaconcepts/nominalizations*.
This last one is especially interesting: academics really do bucket their ideas using certain abstractions, but forget to unpack them for the reader. Plus, English makes it easy to make nouns out of verbs: instead of *postponing* something you can implement its *postponement*, and so on. The example from Explanation 2 above has many instances of this, as does this:
> Prevention of neurogenesis diminished social avoidance.
rather than
> When we prevented neurogenesis, the mice no longer avoided other mice.
The point is that the authors may actually be thinking in the former terms and categories, and fail to write in "classic style" for the reader from the reader's point of view. I believe this is important enough to count as a separate **Explanation 7**.
And regardless of whether authors are thinking/writing in classic style or self-conscious style, it seems to me that many of these bad habits are things one can easily pick up from reading other academic writing, which is **Explanation 8**.
---
In the articles included in the same download, there is an overlapping explanation (**Explanation 9**): academics have many reasons for using jargon, and not all of them are good for the reader:
> Academics turn to jargon for a wide variety of reasons: to display their erudition, to signal membership in a disciplinary community, to demonstrate their mastery of complex concepts, to cut briskly into a continuing scholarly conversation, to push knowledge in new directions, to challenge readers’ thinking, to convey ideas and facts efficiently, to play around with language.
> (Helen Sword, _Inoculating Against Jargonitis_)
----
The other article ([_The Science of Scientific Writing_](https://www.americanscientist.org/issues/pub/the-science-of-scientific-writing/99999) by Gopen & Swan) is very valuable on some precise ways in which academic (especially scientific) writing is harder to understand than it needs to be. Its main point is that there are certain structural cues that readers unconsciously use for interpretation, but authors do not always satisfy those expectations. (Some of its points are also in [this video](https://www.youtube.com/watch?v=jLPCdDp_LE0), accompanied by [this handout](https://www.cmu.edu/student-org/pcr/media-files/pcr-sessions/swan_summary.pdf).)
For example, they consider the following paragraph:
> The smallest of the URF's (URFA6L), a 207-nucleotide (nt) reading frame overlapping out of phase the NH2-terminal portion of the adenosinetriphosphatase (ATPase) subunit 6 gene has been identified as the animal equivalent of the recently discovered yeast H+-ATPase subunit 8 gene. The functional significance of the other URF's has been, on the contrary, elusive. Recently, however, immunoprecipitation experiments with antibodies to purified, rotenone-sensitive NADH-ubiquinone oxido-reductase [hereafter referred to as respiratory chain NADH dehydrogenase or complex I] from bovine heart, as well as enzyme fractionation studies, have indicated that six human URF's (that is, URF1, URF2, URF3, URF4, URF4L, and URF5, hereafter referred to as ND1, ND2, ND3, ND4, ND4L, and ND5) encode subunits of complex I. This is a large complex that also contains many subunits synthesized in the cytoplasm.
If you asked people why it was hard to read, most would mention the technical vocabulary and background knowledge required. These are obviously necessary, so one might consider such paragraphs impossible to improve. However, that is not the whole problem.
At the level of sentences, the article discusses some rough principles:
- In the first sentence above, there are 23 words between the subject ("The smallest") and the verb ("has been identified"). Readers tend to treat what comes between them as an interruption, and pay less attention to it.
- A sentence is expected to make a single point, or serve a single function. It is best if this appears at the end of the sentence (the "stress position").
- At the beginning of sentences (the "topic position", what the sentence is about), the reader expects perspective/linkage and context.
- So, readers are helped most when sentences consistently start with old information (linkage) and end with emphasis on new information. “In our experience, the misplacement of old and new information turns out to be the No. 1 problem in American professional writing today.”
(The above is a crude summary; you can read the article for nuances.)
Based on some of these principles (and with some guesswork about the authors’ intentions), it rewrites the above paragraph to:
> The smallest of the URF's, URFA6L, has been identified as the animal equivalent of the recently discovered yeast H+-ATPase subunit 8 gene; but the functional significance of other URF's has been more elusive. Recently, however, several human URF's have been shown to encode subunits of rotenone-sensitive NADH-ubiquinone oxido-reductase. This is a large complex that also contains many subunits synthesized in the cytoplasm; it will be referred to hereafter as respiratory chain NADH dehydrogenase or complex I. Six subunits of Complex I were shown by enzyme fractionation studies and immunoprecipitation experiments to be encoded by six human URF's (URF1, URF2, URF3, URF4, URF4L, and URF5); these URF's will be referred to subsequently as ND1, ND2, ND3, ND4, ND4L and ND5.
This still has all the jargon, but readers who understand it are more likely to arrive at the same interpretation as what the author is trying to say, and more easily.
But many authors consistently violate these principles and do the exact opposite (putting new information first without linkage, and old information at the end where it receives emphasis), for which the article has the following (continuing my numbering in this answer) **Explanation 10**:
> The source of the problem is not hard to discover: Most writers produce prose linearly (from left to right) and through time. As they begin to formulate a sentence, often their primary anxiety is to capture the important new thought before it escapes. Quite naturally they rush to record that new information on paper, after which they can produce at their leisure contextualizing material that links back to the previous discourse. Writers who do this consistently are attending more to their own need for unburdening themselves of their information than to the reader's need for receiving the material.