10

This is a question from a newbie doing reviews for journals/conference papers.

What should one do if they are among the panel of reviewers for papers for a specific journal / conference, and you get a paper whose subject material you are not an expert in, or are not fully aware of, all the details around which the paper focuses on?

Is it the norm to state that you are not versed well enough in the particular area, thus you cannot make a qualified opinion, or should you just do as best as you can to review what you understand of the paper, and hope that the other reviewers will do a better job, compensating for your weakness? Are there any other options?

10

First off, "fully" is a very relative word, somewhere between expert and novice which makes a specific answer difficult at best. Editors try to identify persons they believe can provide a good constructive review of the manuscript in question. If that is the case, you are viewed as having that expertise. Note that it is not uncommon that an editor appoints experts with different specialities to cover different aspects of a manuscript. It is the responsibility of an editor to select reviewers with care to make sure a manuscript is scrutinized fairly and by peer "experts".

Now this system is not fool-proof and it is therefore the responsibility of a reviewer to decline to review if they think they are not in a position to take on such a review. There are of course many other reasons to decline but that is a different story. So in your case, you need to figure out based on the information you have received, if you have the background to provide input on all or significant parts of the manuscript in the request. When taking on the first reviews in a career, you may ask your advisor or peers about the task but remember, the fact that you have been asked is not necessarily something that is open information (in for example double-blind reviews where anonymity is requested).

So, think about how you can contribute. If you do not see that you can provide input, decline. Otherwise, take on the review. Reviewing is an integral part of academia and getting started is necessary at some point. It can also be quite rewarding since you can gain insights ito new science as well as ways to (or not to) disseminate science.

  • 1
    In any case, it would be good to notify the editor of the parts you are not expert in, so they can hopefully fill them with another reviewer. I remember some medical experiments with huge mistakes in the physics of the problem; and in biology is not rare to encounter sub par statistical analysis. (And who knows what errors I make in the biological part). – Davidmh Nov 5 '14 at 10:21
3

For a multidisciplinary journal or conference, I will sometimes deliberately assign a paper one reviewer who knows significantly less about the subject. The intention is to have a slightly more detached perspective who can say whether this paper is interesting and intelligible to anybody outside of its narrow sub-sub-sub-field. It's also rare to review a paper that you are perfectly knowledgeable about, since science has so many different aspects.

What you should do when you do not perfectly understand a paper:

  1. Be extremely clear on which parts you are confident that you understand and which you don't.
  2. Do not assume the authors are wrong if you don't understand. It may be one of the gaps in your knowledge.
  3. Do not assume the authors are right if you don't understand. They may be blowing smoke in your eyes.
  4. Explain what, if anything, you found of value in the paper despite your lack of knowledge.

If your review, in combination with the others, is not sufficient, it is the responsibility of the editor or chair to obtain another. It is not your job to determine how the reviewers are distributed.

That said, if you are completely and totally lost, contact the editor / chair who assigned the paper to you and check with them. Depending on their intent, they may take you off the paper, or they may tell you that this is exactly what they want you to write down. If you need to do this do it soon---it's terrible form to screw up somebody's reviewing schedule and either create a last-minute crisis or an extra delay for the authors.

0

An editor making a review assignment will usually make sure that you are an expert in the field before requesting you to do the review. However, many times the editor will not know your work personally and will rely on a data base where your set of skills and areas of expertise are listed.

For regular journals, you are given the opportunity to decline the review. You can do this for a number of reasons, most commonly you're too busy or not an expert in the field (if I get a review request from a crappy "predatory" journal I just ignore the email). The editor will usually then ask you to propose alternative reviewers that might be interested in acting as referees.

In special cases, such as conferences, authors might be expected to act as referees for the other attendees. This is an ethical commitment since you should correspond to other authors who spend their valuable time reviewing your paper. The editor will again try to ensure the paper sent to you falls within your field of expertise. However, if you are not an expert and feel that you will do a bad job, it is not fair to the authors that you accept the review. Contact the editor instead to point out the issue and ask him/her to get another paper more within the area in which you're knowledgeable.

0

For most journals and conferences that I've reviewed for, I had to give at least two grades: one for overall (something like 1=clear reject up to 5=clear accept) and one for how well I would rate my own expertise of the subject. Sometimes there's also other grades like how well is the paper written etc. In this case, the question answers itself. Otherwise, there's usually a "notes to reviewers" section where you can comment that you're not an expert in the area.

I would only decline a review due to not being an expert if it is a journal or conference specifically aimed at people who are experts in the area - in which case, whoever asked me to do the review has made a mistake in nominating me.

Otherwise, I think one of the things a journal is deliberately looking for is how well the authors can explain and present their work to a non-expert in the area. Even if you don't get all the details, you should judge how well the paper gives you a general idea of what the authors are doing, how it relates to previous work, what the novelty of this particular paper is and why the subject is important etc. In my opinion, the best review panels contain at least one expert (who will be able to comment on details) and at least one non-expert (who can focus on the bigger picture).

0

Virtually all review forms that I have come across so far featured a field such as Reviewer confidence or Knowledgeability of reviewer, which was meant for exactly that purpose - to state how confident you feel in the respective topic.

What will be done upon that field will be up to the program committee:

  • With too low a confidence, the program committee may decide to give your review a lower (or, in extreme cases, a very low, down to zero) weight.
  • On the other hand, the goal might be to intentionally involve a mixture of differently knowledgeable reviewers.
  • Based upon the reviewers' self-assessed expertise, the program committee might also decide to involve another, possibly more knowledgeable, reviewer.

Your self-assessed confidence level will often be cross-checked by a question asking you to explain the contents of the submission in your own words. That helps further to determine how and what parts of the submission you understood.

So, in short: Expressing how knowledgeable you are in the field of the submission is expected by program committees, to a point that often, a dedicated form field is provided for that information.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.