22

I am reviewing a paper for a major journal with very high impact factor. Some of the authors are very well known in the field and have several other papers published in this particular journal, such that their objectives and results are likely valid ("likely" is the best you can get in this field, so don't get hung up on this). However, they are not native English speakers and the paper is almost completely unreadable.

Usually, I would immediately reject the paper. Due to their reputation, however, I forced myself to stay with the paper for a couple of weeks. From what I could understand after several attempts, their results might indeed have the potential to be published in this journal, but the editorial board is typically very strict: If I send a report claiming that the paper needs to be pretty much fully rewritten, they will likely reject it. I do need, however, a fully rewritten version even to be sure that I properly understood what they claim, and then to reinforce or change my mind about the paper's relevance.

I am thinking about sending a report where I state what I believe the paper is about, and if this is the case then it might be relevant, but I cannot be sure because it is currently unreadable. If the authors revise it, then I will be happy to read it again. The journal might still reject it, and I'd feel bad because this is essentially blaming people for not knowing English. I then wonder if there is any other way, e.g. contacting the editors directly instead of writing a report. Would this be better? Are there other options?

9
  • 4
    If they reject it, is there a chance to resubmit?
    – Sursula
    Oct 9, 2023 at 6:33
  • 28
    Re: fairness, many native speakers write unreadably and need editors too :) Oct 9, 2023 at 14:15
  • 14
    You shouldn't feel bad about "blaming people for not knowing English", because you aren't doing that. The editorial board might be (by rejecting a paper just because the English needs extensive revision) but that is not under your control. Oct 9, 2023 at 14:53
  • 28
    Seems like an open-and-shut case of "revise and resubmit" to me.
    – Max
    Oct 9, 2023 at 17:54
  • 6
    @LukeSawczak And on the flip-side, many non-native speakers have better command of the English grammar than native speakers.
    – EA304GT
    Oct 9, 2023 at 18:21

7 Answers 7

47

Reviewer bias in single- versus double-blind peer review

Scientific peer review has been a cornerstone of the scientific method since the 1600s. Debate continues regarding the merits of single-blind review, in which anonymous reviewers know the authors of a paper and their affiliations, compared with double-blind review, in which this information is hidden. We present an experimental study of this question. In computer science, research often appears first or exclusively in peer-reviewed conferences rather than journals. Our study considers full-length submissions to the highly selective 2017 Web Search and Data Mining conference (15.6% acceptance rate). Each submission is simultaneously scored by two single-blind and two double-blind reviewers. Our analysis shows that single-blind reviewing confers a significant advantage to papers with famous authors and authors from high-prestige institutions.

You're describing something that double-blind review is supposed to prevent. If the paper is unreadable, you should tell the editor, regardless of who the authors are.

I then wonder if there is any other way, e.g. contacting the editors directly instead of writing a report. Would this be better? Are there other options?

The result would be the same - the editors receive a "there's potentially useful science here, but the English is incomprehensible" response from you, and then it's up to them to decide what to do (remember editors can choose to accept a manuscript which reviewers recommend reject for). Practically speaking the difference if you write to the editors directly is that they'd have to submit the recommendation for you, which is a minor administrative hassle, but still an administrative hassle.

12
  • 11
    @QuantumBrick but if they are incapable of bringing their paper to a readable and understandable state, then it really should not be published and it is correct to suggest rejection.
    – Sursula
    Oct 9, 2023 at 8:53
  • 11
    @QuantumBrick "Good for them" is not the point. The point is "bad for everybody else", everybody who you would not have given this special accommodation. Out of fairness, you should treat everybody the same. Oct 9, 2023 at 10:30
  • 13
    @lighthousekeeper You don't deal with near-irreproducible experiments without taking a look at who wrote the paper. The double-blind vs single-blind thing is cute until you meet with people claiming extraordinary effects due to elaborate experiments that can only be performed by a couple of groups in the world. You treat a paper submitted by a group with a solid reputation very differently than from a group no one has ever heard before - and you should. What people have to understand is that not all fields of science are the same, and what people refer to as "scientific method" is very fluid. Oct 9, 2023 at 11:20
  • 16
    @QuantumBrick You are conflating two separate discussions - one is about trust, the other about the bar for an acceptable paper. The latter should never depend on the involved people. There are also issues with the former, for example, cases of high-profile researchers being found out as fraudsters -- after all, a high-profile researcher might feel a particularly high pressure to continuously deliver astonishing results, and this could be even more harmful in an area dealing with near-irreproducible experiments. Oct 9, 2023 at 13:43
  • 5
    @QuantumBrick "given that I have read the paper, which I would not have read if I didn't know the authors, I have come to the conclusion that it might actually fit this journal" If it's a good, on topic paper with interesting ideas, then it's all the more tragic that nobody who reads the paper will be able to understand those ideas because it's unreadable. Tell them as much so they can revise and resubmit.
    – Ray
    Oct 11, 2023 at 15:53
32

Stop wasting your time. It takes at most 30 min to find out that a manuscript is "unreadable". Why did you spend weeks with this? That is a disservice to yourself (and your students and your employer and ...). It is also a disservice to the authors, who have to wait for a decision where the editor should have done a desk-reject.

6
  • 2
    Underrated comment IMO. Better for the authors to get rejected quickly, be able to send it to a language editing service, get their help to improve it to a readable level, and then submit it elsewhere. Oct 10, 2023 at 8:12
  • 3
    Because my report is not the only one. There are usually three reviewers and the average time to get all three reports is 1.5 to 2 months. I tried reading the paper several times because I work exactly in the same field and have access to this paper, which is not public, and it is therefore a good opportunity to get familiar with what this group is doing. Your comment is almost useful, but all of your points have obvious answers and you would have posed them in a more polite manner had you been aware that there are other fields in the world and you might not know how they work. Oct 10, 2023 at 13:22
  • 5
    I do not agree with your comment and you don't appear to understand my answer. If a manuscript is "unreadable" there is no justification to spend time and effort as a reviewer. (Also, I'm German. We are very forthright and some cultures seem to confuse that with being impolite or rude.)
    – user9482
    Oct 10, 2023 at 13:33
  • 8
    @QuantumBrick If the paper is unreadable there is a good chance the other reviewers have already said so, and your report is the only one left. Average times are only relevant when considering average papers. Oct 10, 2023 at 15:29
  • 2
    @Roland It was worth deciphering the paper. I can assure you. Btw: I have lived in Germany for a long time, and where I come from Germans are considered too polite. It is just that telling someone they are disservicing several people without knowing what is going on is not a display of straightforwardness, it's just void bluntness. Oct 10, 2023 at 15:50
22

Converted from a comment by another user:

You can of course always contact the editor. However, precisely because it is a high IF journal, it is probably read by many people. Inflicting a "likely" correct and interesting but unreadable paper on them is a disservice. Research is communication, and the accepted language is usually English. You are not blaming people for not knowing English. You are performing a valuable gatekeeping service. The editor might suggest to the authors to have their next paper proofread before submitting. This is unfair to non-native speakers, but sometimes there is no "fair" solution.

I share this opinion. Just let the editors know your honest opinion on the paper; it is up to them to decide what to do. Even if the paper is rejected, this is not the only journal in the world.

I suggest not to contact the authors independently: this would break reviewer anonymity, which is an important safeguard to have (especially in cases like this one, where you are essentially giving a negative opinion on a paper by a well-known author).

8

It isn't your job as a reviewer to try to decode a badly written paper, nor to second guess editorial responses. Write an objective review saying what you said in the question "potentially interesting but almost unreadable" to the editor. What the editor then does is not your problem.

2

If you do not understand what the paper is about, do you think one single journal reader will? And certainly none of the readers will try for several weeks to understand a single journal article. If they do not understand it, they will put it aside.

So what is the use of accepting and publishing a paper nobody will be able to understand?

Tell the publishers that the content seems to be interesting and worth for publication, but it has to be rewritten.

1

Easy. Write some version of what you just did here. Say something along the lines of, this work is technically competent, addresses an important issue, is an important observation for the field, comes from a group who have made many important contributions, etc etc. You can then justify your statements with the scholarly details. You then state that the one major weakness of the paper is that the standard of written English is very poor; so poor that it took you a long time to get through it. (Don't say it took weeks, though). I serve on the editorial boards of several journals and I would not reject a paper on that basis. I make it very clear to the authors that they need to get help from somebody who can actually write in understandable English, and I give them at least one shot at fixing it. On most online review websites there is a place where you can send a message that only the Editor will see. If you're really concerned about this you can write there that you really want to see this paper get published, but that the English is awful.

1

Ah, the good times of reviewing papers... I miss none of them, so much painful reading. I am not going to provide any particularly smart or new answer, but I feel that sometimes we get so hung up into complex considerations and we don't see that the best approach is simply to go back to the basics. What always worked for me was to begin by asking myself the most basic questions. Do the authors make the problem they are facing clear and the contribution of the paper in that respect?

If this is already hard to detect, it's a rejection. If these points are reasonably clear, then proceed to the second level: do the authors clearly explain their procedure to arrive at those results and provide a (reasonably) fair analysis of their importance? These two questions are harder to answer as they usually involve some very in-depth specific knowledge in areas in which the authors themselves might know more than anyone else.

Even in that case, though, the path for a reviewer is not too difficult. In my opinion, one should always keep in mind that a paper is not the "discovery" itself, but it's an instrument used by the authors to communicate the discovery to others. So, in a sense, evaluating it should not be thought as conferring or not some honors to the authors, but rather as maximizing the benefit for the readers. If the paper does not fulfill that function because is unreadable (I would say for the average reader of that journal, a criteria which might be possibly relaxed by considering a more restricted group, depending on the subject), I think one should definitely ask to resubmit.

Remember, if you do that, you are not dissing the authors, you are just helping many readers learn something potentially important without going through the same pain that you experienced. In a sense, you should see yourself as the hero who takes one for the benefit of group.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .