During peer review, how to handle comments that we are not able to address?

Question

We have submitted a marketing paper about consumer responses to service failure in online reviewers while reading. This is a second review round and the comments makes me upset.

1. We have reported age, education, race, occupation and gender. We didn't ask for marital status and income. The reviewer said we didn't provide enough details on the demographic.

2. We have research questions for the explore analysis and the reviewer didn't accept research question (RQ) and believe all studies should use hypotheses.

3. We have clarified that pilot and main study are two independent samples in the first review round and added notes towards it in the manuscript. The reviewer still asked us whether the pilot study will impact the participants in the main study (I don't understand this question).

4. The reviewer said we have to have a screening question to exclude those who have never experienced a service failure in their life. First, this study has already been done. Second, whether they experienced it or not does not matter for our study as we used scenarios to described the service failure rather than let them recall their own experience.

From my understanding, only the 4th one can be explained （but is still very unprofessional), all other comments are not reasonable.

Can we write an appeal to editor or what to do?

Answer 1

You can't appeal a review, as a review is not a decision. The reviewer only advises the editor, and the editor then makes a decision. Assuming this is not a rejection by the editor, then what you should do is make changes that are reasonable, and resubmit those changes. With that you add a response where you point by point clarify what changes you made and what pieces of advice you did not implement and why. Keep that very professional and very neutral. Do not use words like "idiotic". If the comments are indeed idiotic, then the editor does not need to be told. Instead it is much more powerful if the editor comes to that conclusion on its own.

A good rule of thumb is to remove all adjectives from your response letter. I make exceptions for some positive adjectives.

As a rule, even comments that make no sense are often useful: It indicates that a reader did not understand your paper. Moreover, you can often deduce what part of the paper was confusing. That is usually something you can fix.

If the review was accompanied by a rejection by the editor, then you could try to appeal. The chance of success is not good. So most often you will loose time during the appeal, and then have to submit somewhere else. So often it is quicker (less glacially slow) to directly submit somewhere else.

Answer 2

You may write to the editor, however, it might not serve any useful purpose.

With that said, what you need is to

1. Critically look at the reviewers' comments with open mind, no matter how 'idiotic' it might appear to you and your team.
2. Prepare a rebuttal addressing the issues raised. You need to be polite about this irrespective of your feelings or views, yet assertive enough to respectfully disagree where need be. Use sound judgement, literature and logical reasoning. Take emotions out.

At times, what we consider useless or idiotic might be trigger to recheck our fundamentals. I'm currently busy with a major revision. At first, some of the comments appears offish and petty. After letting it lie low for about a week, I've come back to it yesterday and continue my rebuttal. Though I don't agree with these comments (the parts sounding offish), I've now touch based with literature again. Where need be, I've adjusted, where not, I respectfully disagree.

Regarding 3 and 4, it might be that the review is concerned about common method bias (CMB) or Common method variance (CMV).

A good literature is Podsakoff & Organ, 1986
You may also read up Malhotra et al., 1986 and Rodríguez-Ardura & Meseguer-Artola, 2020