Does using complex mathematical notation detract the reader from a paper?

Question

I'm currently writing a paper on analysing miRNA targeting in animals. It's principally a statistical analysis involving many graphs.

However, to explain the creation of one of the graphs, I've resorted to using set theory notation, as this, to me, seemed the most efficient way of explaining how the graph was produced.

However, this may potentially have some issues:

1) Will the use of terminology that is outside the theme of papers in this field detract from the description? Such that few people who read this paper will understand set theory and therefore many will not understand the mathematical notation? The description, I hasten to add, will be a tenth of the length when using set theory notation.

2) Evidently, concepts should always be written as clearly as possible, but where is the line formed between explaining concepts in words, as opposed to mathematical notation? Why would one choose one way over another?

3) Should you make assumption of the mathematical understanding of a person reading the paper?

Answer 1

An author should always be careful when using mathematical notation that is "outside the theme of papers in this field," especially if the papers typically have little math notation. I have personally found that running into a cryptic (at first glance) equation makes me put the brakes on and engage some deeper thought to decipher what's trying to be said. Oftentimes, the notation is required, or at least useful, but when I end up thinking, "Couldn't they have just said 'such-and-such' in plain English'", I get a bit cranky.

Sometimes, though, you do need the space savings to stay within page limits, so if you really will shave off 90% of the space, and you need that space to fit in something else that is key to the paper, AND in your best judgement that the audience won't be derailed by the notation, you might be OK. Otherwise, because you have stated that you can describe it clearly in words, you should consider perhaps rewriting that description to be more concise and precise, rather than potentially obfuscating the details of your graph production.

Answer 2

So far, I'm missing the option of giving the 110% version of set notation plus explanation in plain English. That way, readers familiar with the set notation can skip the written explanation, and others can either skip the formula or learn what it means from your explanation.

(As you think whether you should give the long explanation, I assume that there are no particular length restrictions.)

I may add that I (chemist/chemometrician) was actually taught that a formula should preferrably be accompanied by a plain text explanation of the idea behind this formula.

Answer 3

By and large, I would say avoid using non-standard presentation techniques in your paper. This includes using non-trivial mathematical notation in a field that generally does not use maths.

Consider that simply by choosing your way of presenting your techniques / results you (a) already lose most casual readers of your paper (i.e., those that don't have to study your paper for some reason, but were just interested in its content), and (b) presumably significantly increase the chance of your paper being falsely rejected by a reviewer who was inexperienced in the used notation, and hence misunderstood some details. Of course, this is not supposed to happen in an ideal world, but we don't necessarily live in such a world.

That being said, I should add that (especially for lower-ranked venues), the opposite of (b) may also happen - you may get a reviewer who does not fully understand your notation, but decides that it looks "sciency" enough and hence accepts the paper (even though, maybe, the content was actually weak). I have certainly seen paper submissions that tried to ride the train of camouflaging weak content with complex notation.