26

I have seen couple of questions related to rejection of submissions to conferences (CS). Coming to my case, this is my third rejection, I agree that first two were fair decisions as the paper was not clear and needed more thorough experiments. However in this third submission I have put in all possible effort to make it the best considering feedback from prior submissions.

I received initial comments from 4 reviewers with no scores and based on their comments it was on borderline. Since there was a rebuttal phase, I have sent a response clarifying issues raised by them. I received my final notification today and it was a 'reject' decision. The final reviews had comments from an additional reviewer. His comments are atrocious and unfair. He pointed out that he is rejecting the paper as some issue X was not addressed at all. However we have addressed X with a thorough evaluation in a subsection and results put up in a graph. I'm sure that this reviewer has not read it properly. If he raised his issue may be on the way we solved X, it would have been more appropriate.

I know that appealing to the PC chairs would be a futile effort but at the same time I do not want an unfair review. This feeling of rejection based on unfair review is depressing me. How do I go about it. I feel the only way out is to resubmit it for another conference in near future.

Please advise.

|improve this question
  • 12
    If the paper was borderline after the four reviews you feel were fair, there's a good chance that it would have been rejected even without the extra negative review. It's unfortunate that the extra reviewer doesn't seem to have read the paper properly but you can't say that s/he alone torpedoed the paper. – David Richerby Jun 23 '14 at 14:23
54

I feel the only way out is to resubmit it for another conference in near future.

That about sums it up. Negative reviews and rejections are hard to swallow. They are common in academia and you cannot let them get to you. The best thing to do is step away from the manuscript. If you need to cry, cry. If you need to scream, scream. After a while, generally for me a week or so, try and look at the reviews again. If they still piss you off, step away from them again. Once you can read the reviews objectively, start to tackle them. This might mean agreeing with the reviewer, or seeing where the reviewer got lost, or that the reviewer is simply a fool. Once you have taken everything from the reviews that will improve the manuscript, file it away and if necessary cry and scream again. Then resubmit the manuscript to the next conference.

|improve this answer
  • 1
    Thanks @Strongbad, I agree that I should vent out my frustration first, next consider only constructive comments and refine my paper for next submission. – kris Jun 23 '14 at 12:23
  • 10
    This is a great answer. It happened to me once that I was so pissed off regarding a strong rejection I believed was unfair. So, I waited for something more than a week to re-read the reviews. Still, I found them to be unfair and badly written. So, I decided to just resubmit the paper to an even more prestigious journal. It was accepted with minor revisions at the first round. – user7112 Jun 23 '14 at 14:06
  • 1
    I really wish someone said this to me at the start of my PhD. Perfect answer. – ham-sandwich Jun 15 '15 at 23:40
  • @kris Unfairness is much present. Usually we are inclined towards sadness, anger etc. But in retrospect those are some of the pearls. When we look at it after a few years, you will see them from a higher plane - no more the tress, but the forest. We will realize there are some new things we learnt which we would not have. And also all the people were also in situations themselves. So, dont forget them so that you are safe. But forgive, understand from the experience. Later may things will all get better. This world is the best pulp fiction. :) – Rahul Jun 10 '18 at 2:35
35

The possibility to have unfair reviews is one of the reasons why you receive multiple reviews. I have received multiple reviews where the reviewers clearly didn't review the paper carefully, and it's indeed quite frustrating.

That being said, when I looked back at the papers, I realised that they were not rejected because of one unfair review, but because I didn't manage to convince the fair reviewers: if you get four accept, then it does not matter if you have an unfair reject. Besides, it could also be the case that you've received an unfair accept review (i.e., someone who accepted the paper without really reading), so, in the end, it might balance things out.

Dealing with rejection is a huge step of the academic process, and as it has been said here in the past, if you never get papers rejected, you are probably not aiming high enough. As long as you have only one unfair review, ignore it, and move on to the next conference (and if you only receive unfair reviews, well, you are probably submitting at the wrong venue).

|improve this answer
  • thanks for your advise. I would like to reconsider if I'm actually targetting the correct venues. – kris Jun 23 '14 at 12:24
  • 20
    "that they were not rejected because of one unfair review, but because I didn't manage to convince the fair reviewers" This! As my mentor likes to say: a paper that does not find at least one champion amongst the reviewers almost always gets rejected. A paper that does almost never gets rejected. – xLeitix Jun 23 '14 at 12:30
  • 3
    It's an eye-opening comment, really. At the end of the day, it is not enough if nobody hates your paper. Somebody needs to love it, and if that is the case, he or she will fight for it, even if other reviewers are less convinced. – xLeitix Jun 23 '14 at 12:31
  • 5
    +1 for Besides, it could also be the case that you've received an unfair accept review... so, in the end, it might balance things out. Too often, people forget that potential errors go both ways, and only see the negative ones. – Bobson Jun 23 '14 at 14:41
  • That's not quite right - you do end up seeing a fair few unfair accepts coming to light, only they do it later, and messier. – E.P. Jun 23 '14 at 15:40
6

As you said, the paper was on the borderline after 4 (fair) reviews. I would advise first to discuss the paper and the results with your supervisor / colleagues, and if you are still determined to continue, rework it significantly, taking in account the criticism of four referees, that you consider as fair. Make sure the result moves far from the borderline (and in the right direction). Then re-think the remarks of the last ("unfair") referee and decide what you want to do with them. Probably, you should not trash them away completely — try to find a motivated comments and suggestions apart from the "unfair" ones.

When a new version of a paper is ready, you may wish to show it to your supervisor and send to several colleagues outside your group asking them for some pre-submission comments — this may save you a lot of time.

|improve this answer
4

Are we talking about the same paper that is rejected a total 3 times? If so, you should seriously consider if this paper (however you improve it) will ever be good enough. Perhaps the problem you solve is not interesting enough or your solution does not promote the state-of-the-art significantly enough. In that case, maybe aiming too high is your problem. Also, note that in prestigious CS conferences (I assume this is your case, due to a rebuttal phase), borderline papers never make it. At the end, there are enough papers who pass with flying colors and the rest fail.

You should start considering submitting to a lower ranked CS conference or perhaps a workshop. You have already wasted a full year or more (that is what probably the time required for getting those 3 rejections)and wasting another 3-6 months for the next conference, might not be feasible. You should seriously start discussing this possibility with your supervisor.

I understand that rejection hurts but many of us had works that were rejected, kept improving it and got rejected again. At some time, it is time to cut your losses and move-on. I think it was @JeffE that said "The universe has spoken. Move on" and he is right. Submit to a lower conference / workshop to at-least patent those results and lower your sense of rejection, but without wasting too much time further improving those results. Instead focus on more interesting variations and move-on. But most-of-all discuss all these possibilities with your advisor

|improve this answer
  • thanks for your comments. The paper was rejected twice owing to writing quality and rigorous evaluation. Overall they liked the idea however. I will try to resubmit for a lower rank conference now. – kris Jun 23 '14 at 12:47
  • 1
    Whether or not there is a rebuttal phase depends on things other than the prestige of the conference. I'm not aware of any conference in theoretical computer science that gives authors the opportunity to rebut reviews. – David Richerby Jun 23 '14 at 14:34
0

I think that the real problem that may however be hard to address, is that there is a schedule to visit certain conferences well in advance and that you are supposed to come up with research results that will be good enough for those conferences. This is not what I am used to doing, but then my field is not CS. If you could simply work on your research program until you get outstanding results, then getting those results published in a decent journal won't usually be a problem. There will still be cases of bad refereeing but then you won't mind if the referee wants things that are already explained to be clarified if the report starts with the statement that the paper is excellent and should be published.

|improve this answer
  • 1
    Not all computer scientists decide which conference to publish in and then find a result to publish there. There are plenty of us who find a result and then decide which conference to send it to. – David Richerby Jun 24 '14 at 11:09
0

In Hebrew there's a saying:

צרת רבים - חצי נחמה

which means

The trouble of many is half a consolation

Unfair reviews seem to be very common. I just got a set of reviews a few months ago with one reviewer saying (roughly) "section X was great but section Y was weak" and "I really liked section Y; unfortunately, section X was sub-par" and they made conflicting suggestions regarding what to expand on and what to mention in passing. And then there are people who seem to just not be close enough to your field of research who review the paper in a way that makes that fact abundantly clear. etc.

So while this answer isn't any sort of practical advice, at least know that you're not alone with this experience.

|improve this answer
  • "[T]his answer isn't any sort of practical advice" - it should really be a comment rather than an answer in that case. – user2390246 Aug 18 '16 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.