5

This is probably more relevant to conference submissions than to journal submissions, but still.

Some CS conferences have recently started to employ a resubmission policy: if your paper got rejected from a prior (relevant) conference, and you are resubmitting, you have the option of attaching your previous attempt’s reviews and a cover letter explaining how you addressed them.

You don’t have to do this. There are very good reasons to believe that doing so is beneficial. For the authors, it offers a chance to show improvement and a willingness to improve (thus deflecting potential repeats of prior criticism). For the community, it prevents p-hacking of the review process (resubmitting until you luck out and get your paper accepted).

What are the potential downsides of making this process mandatory?

Improve this question
  • 10
    The title and the body don't particularly match. – Jon Custer Sep 4 '19 at 14:17
  • Related: 1) Submitting a paper to a new journal with responses for the rejection to the first journal, 2) Are you permitted to forward the reviews from a rejected manuscript when submitting to a new journal? – Anyon Sep 4 '19 at 15:05
  • 2
    That is not "p-hacking." Or hacking of any kind. – Anonymous Physicist Sep 7 '19 at 11:26
  • I have seen authors resubmit the exact same paper to related conferences four or five times. I was involved in the review process of that paper in varying capacities, and every time it got negative and constructive reviews. The authors kept the typos. I don’t know what to call it but it’s definitely not something I want to see. – Spark Sep 7 '19 at 11:47
  • And I have seen program committees rejecting papers multiple times, based on positive reviews, acceptance recommendations, etc., with some minor or even no substantial criticism. I don't know what to call it. Nepotism? Cliquish behavior? But it's definitely not something I would like to see. PC should explain clearly and justify their decisions, or admit that the decisions are arbitrary or utterly subjective. (Of course, I agree that constructive criticism by reviewers should be taken seriously!) – Dilworth Sep 7 '19 at 11:57
1

The obvious downside is that you can't verify it. You could have a policy which says "if your paper has been rejected elsewhere, you must attach your previous review and a cover letter explaining how you addressed the issues". However if the authors don't do it anyway, how would you find out? The only chance is to invite either the editor or reviewer(s) who reviewed the previous submission. The odds of this is small, especially since you don't know who edited/reviewed for the other journals, or even which journal it was.

The other danger is that you open a potential path for fraud. Currently the most common form of fraud is for authors to suggest as reviewers people who are linked to them (or to themselves using obscure email addresses). With this policy, one could easily write the "previous review" oneself to give the paper a sheen of reputability. The new conference cannot verify these previous reviews are legitimate. In principle this should not fool the editor, but in practice it is all too easy. Example of the former ploy fooling journals; this is likely to manage as well (at least some of the time).

Improve this answer
  • You don't have to verify each submission. But every case of proven fraud will be punished by expulsion and public reprimand. – aaaaa says reinstate Monica Sep 5 '19 at 3:44
  • 3
    Actually, the chance that there is some overlap in the program committee between two conferences in the same area is not exactly low. So you could detect it in most cases. However it puts another burden on PC members which isn't necessarily something you want to do. – Maeher Sep 5 '19 at 7:59
  • 1
    To follow @Maesher's comment: Some groups of related conferences plan overlaps between PCs exactly for this purpose. In theoretical computer science, every STOC program committee has someone from the previous FOCS program committee, and vice versa. (STOC and FOCS happen every June and November, respectively.) – JeffE Sep 5 '19 at 11:45
  • In this case though, wouldn't submitting the review achieve little? The editors can already access the old reviews. – Allure Sep 5 '19 at 13:11
1

To complement the other answers, another downside of requiring the disclosure of a previous rejection is that difference conferences have overlapping, but not identical interests. One conference values theoretical results more than experimental results on proof-of-concept implementations of new ideas, while for other conferences, both may be mandatory.

Hence, if a paper is rejected because it's just below the threshold for acceptance, it is quite common that another conference with a different focus will accept it. The previous reviews may however not actually reflect that this is the reason for rejection, as reviewers often focus on the points that they deem important. Then the paper would have a harder time getting into the second conference because it looks much worse than it is.

Also reviewers are sometimes simply wrong with their judgement - they may claim certain parts of the paper to be incorrect even though they are. That puts the authors into a defense position, which is an additional burden on them.

Improve this answer
1

The downsides for this process are clear I believe: a haphazard and arbitrary initial rejection of a paper will play a significant role in forming a biased opinion in the minds of the program committee. Instead of looking at the paper with fresh eyes, they will conveniently reject it again based on a prior possibly mistaken opinion (they do have other hundreds of papers to sort out, so eliminating quickly one paper is only convenient for the committee).

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.