What is required of a mathematics referee?

Question

To review a paper that was submitted to a good journal, what am I required to do? Must I understand everything presented in the paper? Must I check the details and verify each argument/proof? Or is it sufficient to check that the paper is generally well-written, contains no obvious mistakes, and is interesting?

Answer 1

My experience as a both reviewer and author in pure math suggests the following:

• there is no universal standard for refereeing (some referees check papers thoroughly, some are much more cavalier)

• editors may have different expectations/desires of reviewers, depending on the situation and on the editor

• the typical expectation is that you assess a paper from several aspects: importance of the paper (this includes factors such as usefulness of the results/methods and novelty), correctness, and presentation/clarity.

• refereeing is a process, not a "yes/no" assessment

• refereeing well takes a fair amount of time, and results in little personal recognition, but is important for the community

Based on your specific questions, here are my suggestions for how you might go about refereeing a paper:

1. First assess the results claimed in their general context. If they do not seem interesting enough for the journal in question, or are clearly wrong, you can simply say that (preferably with more details) and recommend rejection.

2. Understand the general argument of the paper. Read the introduction in detail, and then try to understand the main points of the paper. (You might not read the rest of the paper in order, particularly for long papers.) Assess the novelty and feasibility of this approach.

3. If all seems good so far, convince yourself of the correctness of the paper. This might entail checking all details, or you might be convinced after only checking certain details. While you should definitely assess the correctness of the result, ultimately the burden of correctness lies with the authors. Of course it's great if you can check the paper completely, but in my experience most referees don't. How much effort you put into this may also depend on the situation (e.g., for a completely novel approach with an amazing result, you probably want to put more effort into checking details).

4. Very often in this process, there will be things you don't understand. There could be a couple of reasons for this: (i) you aren't an expert in the methods being used, (ii) the authors' explanations are lacking. In both cases, you can simply ask the authors for more explanations or explicit citations of the results they are using. It's not necessary to understand everything for a first report, but hopefully you will mostly understand the paper after a successful refereeing process, which may include several revisions. However, if (i) is a very serious issue, you might consult with the editor. E.g., if you can read though the details in Parts I and III but Part II is a complete mystery to you, the editor may find someone else to referee Part II. (Possibly this is a conversation you could have when agreeing to referee the paper.)

5. In the above process, you'll likely come up with a list of questions/suggestions, both about the mathematics and the writing. While you should of course include these, these are typically of secondary importance in the referee report (though sometimes if you have some really good suggestions, it can dramatically improve the paper).

Answer 2

The following is all highly dependent on the field being mathematics or something closely similar. It makes no claim about any other fields.

Sorry, but if that's all you do, then your reviewing career is likely to be short, ending (with that journal) the first time you approve a paper that is revealed to have an error that you should have caught but did not. That journal at least is unlikely to consider you in the future. If you have the same attitude generally then other journals will also drop you early.

In and attempt to avoid further misunderstanding, let me try to be more clear. In mathematics we really really like published material to be correct. But people do make mistakes. Reviewers serve as a bulwark against poor work that looks superficially good actually making it in to print. But unless you take the job very seriously, as mathematics, you are highly likely to miss errors that should be caught in the review process.

You are the last real guard against errors getting published. You may also be the first independent look at a paper other than the author(s). And the errors can be subtle. Errors also reflect on the reputation of the journal as well as the authors. That makes editors very unhappy. Editors want a stable of reviewers they can trust to get it right. That means, first, that you can assure them that papers are correct, to the best of your ability to determine. They also need some assurance that you actually have the ability to determine correctness and treat the job seriously.

So, if an editor sends you a paper and you treat the reviewing task casually (just "generally well written" and "no obvious mistakes") and you miss an error that should have been caught then that editor, and by extension, that journal, won't be well pleased and you are unlikely to be offered additional papers. Of course you are generally anonymous to everyone else, but when other editors, from different journals, give you a try and you also do the same, then your opportunities decline, journal by journal.

If you don't completely understand the paper and its arguments then you are probably not the best person to review the paper. In math, which is very balkanized, this can be a problem. It has been around a hundred years since any individual was able to understand all of mathematics. Editors will, therefore, try to send papers to people with the same specialty whenever possible. Mathematics is subtle and deep. Editors expect high standards both from authors and from reviewers.

The writing style and presentation are important, but not nearly on the same level as correctness. And even with diligence, errors do pass through the system and get published.

If a paper is so poorly written that even a field specialist can't follow it, then it needs a serious re-write at a minimum and it may be hiding serious errors.

I have had to turn down editors for papers outside my current interests as I felt that I was unable to verify the claims and all of the steps in the proofs. Neither was it fair to the authors to spend the time to come up to speed in the area covered by the paper as it would delay publication for too long. Best for everyone that I just stepped back and passed the job to someone more current in that field.