8

Many conferences in my field use two-phase reviewing: in the first phase, 2-3 reviewers review the paper; their reviews are made available to the authors; the authors are given a chance to respond; and then in the second phase, the reviewers read the authors' response and the program chairs might optionally solicit additional reviews.

If a paper receives poor reviews after the first phase, so that it looks like the paper is likely to be rejected, is it ethical to withdraw the paper and submit it elsewhere, without making significant changes?

I read Ethical implications of withdrawing a paper during the rebuttal phase and submitting it somewhere else, which asks about a similar question, but I am interested in the specific case where the authors were well-intended (they submitted to the first conference with the legitimate hope it would be accepted) but do not plan to make significant revisions before re-submitting -- a case that is not covered by the answers there. For instance, it is not uncommon to submit to a selective conference in hopes that it will be accepted, discover that the reviewers don't consider it strong enough for publication there, and then consider submitting to a less selective conference. (So it's not that the paper is flawed, but it isn't strong enough in its current form.)

Improve this question

3 Answers 3

Reset to default
6

It depends on the nature of the feedback.

Clear no: If the reviewers noted severe shortcomings such as objective technical flaws or omissions of significant related work, not addressing the feedback seems a clear-cut ethical violation, as the authors would now misrepresent the soundness/novelty of their work despite their better knowledge.

Gray area: If the reviewers struggled to understand the paper due to presentation issues, one might argue that not addressing this feedback would lead to a wasteful use of a scarce community resource (reviewer time).

Clear yes: A type of feedback that would not require addressing is non-actionable, subjective feedback ("Your technique only does X, but I would prefer to see a technique that does Y").

Improve this answer
8
  • 8
    @NateEldredge The ethical violation doesn't arise from the withdrawing, but from the unwillingness to address the feedback (as far as the feedback is actionable). It's actually the same situation as if the authors just waited for the paper to be rejected and then resubmitted without addressing the feedback (which I would consider unethical as well).
    – lighthouse keeper
    Commented May 24, 2020 at 17:32
  • 3
    Okay, I see what you're saying. But I think I would only say that the authors have an obligation to seriously consider the feedback, and decide for themselves to what extent it ought to be addressed. It is after all their paper, not the reviewers'.
    – Nate Eldredge
    Commented May 24, 2020 at 17:41
  • 1
    @NateEldredge "It's their paper, not the reviewer's" - yes, but it's the reviewer's time that is being wasted. I quite agree with the current answer, though, good classification. +1
    – Captain Emacs
    Commented Aug 8, 2021 at 14:56
  • 2
    @Buffy The relevant category of ethics would be professional ethics. In the case of computer science, a relevant document is the ACM code of ethics. In the "clear no" cases from my answer, re-submitting the paper without changes would violate the principle "1.3 Be honest and trustworthy".
    – lighthouse keeper
    Commented Aug 8, 2021 at 19:45
  • 1
    @Buffy See the two cases in my answer. If the reviewers make the authors aware of a flaw in a proof or of critical omissions of related work, re-submitting without fixing is dishonest.
    – lighthouse keeper
    Commented Aug 8, 2021 at 19:59
4

There is absolutely no ethical concern or constraint at all here. The reviews you get in academia for such works are a gift to the community and to you. You have complete control over the paper until you give up your rights to it. The reviewers aren't paid by the conferences (or journals). If I as a reviewer help you improve a paper you have submitted to conference A and it later appears at conference B instead, everyone has gained. And you aren't bound to take my advice. There is no contract to that effect. If I review your paper, I get no rights to take over its content or ideas.

Moreover, it may be that the paper without change is actually appropriate elsewhere and the reviews were colored to some extent by the nature of the conference.

Most academic conferences aren't held as profit making concerns and nearly everyone involved is a volunteer (some support staff may be paid). It is a service we do th the idea of scholarship itself, nothing more.

If I give you a gift, it is yours to use as you like or not and welcome. You have taken nothing that wasn't given freely.

Over the course of your life you have gotten a lot of advice. Some of it you even solicited. But much of it you probably ignored. It was advice, nothing more. Use it if it is valuable.

While there are no ethical concerns, there are some practical ones. The sets of reviewers in the various CS conferences overlap somewhat. And if you got particular reviewer for one, there is a non-zero chance the same person will come up again, especially for topics with few available reviewers.

Improve this answer
9
  • 3
    Of course the work of the referees is not "free", it was paid by their home institutions which consider this activity a part of researcher's job. But it's actually irrelevant who is paying as long as it's not the authors. Wasting resources you've paid for may still have ethical concerns, but wasting resources resources you've got for free - definitely does.
    – Kostya_I
    Commented May 25, 2020 at 12:02
  • 3
    so, in these 40 years of teaching, you never wrote a review in your office during your work hours, only in your spare time? And if you did, it would be considered a breach of job description and you could be fired for that? Of course not. When reviewers "voluntarily donate their time", they in practice donate their work time, paid by employers.
    – Kostya_I
    Commented May 25, 2020 at 13:17
  • 3
    As for your analogies, you have a "gift" analogy, that uses someone else's resources but is unsolicited, and "advice" analogy that is solicited but typically requires no resources. But in the refereeing situation, it's both. By submitting to a conference, you solicit referee report and thereby commit not to waste resources put into it.
    – Kostya_I
    Commented May 25, 2020 at 13:26
  • 4
    If you like analogies, suppose you have a non-native English speaking friend who asks you to check language in their paper. After you do it (for free!) they decide not to correct the mistakes you have pointed out. I doubt you would be happy about the situation, and willing to do it for the same friend the second time.
    – Kostya_I
    Commented May 25, 2020 at 13:30
  • 2
    DV because reviews aren't gifts. They may impose an obligation, namely whenever they point out serious flaws. (Gifts, by definition, impose no obligation.) Sweeping known flaws under the rug contradicts sound science, contributes to "publish or perish" culture, and is therefore "unethical" (in the loose sense of the word often adopted on this site).
    – henning no longer feeds AI
    Commented Aug 8, 2021 at 10:01
3

I am a reviewer for a top-tier conference in my field to which thousands of submissions are received every year. It is NOT ethical to resubmit without revisions when you know from the reviewers that the manuscript has flaws and needs further work. The review burden is really heavy on us and we do it for "free", not as a gift to you as @Buffy said, but for our hope to "help improve science". If you take the reviewers' advice, ignore it, and submit the same manuscript somewhere else, you have (1) wasted reviewers' time and added an unnecessary burden to their already hectic life, (2) betrayed your conscience as a scientist whose main goal should be to build and improve upon our current knowledge. I have noticed, in many cases, all it takes for the authors to improve their paper is to spend a few hours of work to improve their benchmarking and analysis. Yet, they opt for resubmission of the same manuscript to a lower-tier conference. That is not helpful to scientific community and certainly not helpful to you as a person to grow as a scientist and reach higher. I suggest you reading this post in reddit for getting the perspective of a reviewer (I am not the author of this post): https://www.reddit.com/r/MachineLearning/comments/fkgfax/d_confessions_from_an_icml_reviewer/.

Even if you choose to withdraw (which I can understand), please do consider the reviews and improve your paper. Don't follow the "publish or perish" culture that is poisoning the academia right now and always submit your work when, to the best of your knowledge, it is complete.

Improve this answer
7
  • Hmm. It's not "ethical" because it makes you very angry? Interesting standard, that. You seem to presume that by reviewing you gain an ownership position.
    – Buffy
    Commented Aug 8, 2021 at 10:56
  • I'll note two additional things. First, a reviewer works for the editor or conference chair. Second, the authors have offered something of value - a potential paper to be published. I think you have the responsibility exactly backwards here. If you don't like the process you can withdraw, of course.
    – Buffy
    Commented Aug 8, 2021 at 12:49
  • 3
    @Buffy: I am not sure what in my post made you assume I was angry (projections maybe?). I made some parts bold for people who want to get the gist of my post without reading everything in detail (I appreciate when people do that too). As for the rest of my comment, I am simply trying to encourage people to devote themselves to high quality scientific work rather than simply publishing. That is exactly why I decided to offer my time for free as a reviewer to begin with. The incentive for reviewing is not like "give and take", but like "I am helping you to do better, so please try your best!"
    – pegah
    Commented Aug 8, 2021 at 14:24
  • 3
    @Buffy Having been quite disappointed lately with the quality of submissions of a conference I used to very much like, I must agree with the responder here. I do not think that authors actively "prefer" to publish low quality work, but that their priority has moved away from publishing high quality work; it has become an afterthought. They want to publish somehow, with whatever ideas/results they somehow cobble together, and the reviewer gets an uninspiring and sloppy paper to review; at least the authors could attempt to incorporate their advice. It's a matter of courtesy if nothing else.
    – Captain Emacs
    Commented Aug 8, 2021 at 15:06
  • 2
    @Buffy I quite agree that reviewers can be blinkered, one-sided, self-serving etc. This is not the case that needs "ethics approval". The thing that worries me about OP's question is "without making significant changes". If there is a substantial weakness, it needs to be addressed. Also lower-tier conferences have a right to get good-quality papers - maybe the topic is less popular, or it has fewer technical requirements, etc. but substantial flaws should be eliminated. If OP believes that the reviews are not substantiated or the request for changes out of scope for OP's work, that's different
    – Captain Emacs
    Commented Aug 8, 2021 at 16:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .