7

Does the submission number in any way influence the way a paper is treated?

For instance, is it likely that an early submission will be picked up by a member of the program committee, rather than being handed over to a subreferee? When I'm going through a list, I do have a tendency to be more concentrated on the first few elements.

Thus, is it a good idea, knowing that I'll be on time for the conference deadline, to submit a good draft a few days/weeks before in order to rank higher in the submission number, then update with a polished version?

  • My short answer: you submit your paper when you're ready to submit it. – scaaahu Apr 29 '16 at 11:35
  • 6
    @scaaahu: Your short answer does not answer my question though :-) – Michaël Apr 29 '16 at 12:05
  • 1
    I have heard of some unofficial research that found that the closer to the deadline a paper was submitted, the more likely it was to be accepted. – stannius Apr 29 '16 at 20:41
19

With a typical peer-reviewed conference, submission number has absolutely no effect on the way a paper is treated. Lots of excellent well-established folks submit at the last minute, and paper assignment is generally not done until the deadline is passed. Furthermore, paper assignment is often done with the assistance of a paper-handling system like EasyChair, which includes randomization and non-order-based heuristics.

  • You are saying that the papers in EasyChair are never presented in order of submission (or any other order, like alphabetical), and that it is unlikely that a program committee member would peak at the submissions before the deadline? – Michaël Apr 29 '16 at 12:06
  • 2
    @Michaël, I'll say it more or less. EasyChair shows reviewers a list of papers numbered in submission order, but the assignment of 6 or 8 out of say 100 papers to a reviewer is randomized (accounting for declared conflicts) in a way that submission order isn't factored in. I don't know of any studies that show a bias towards lower-numbered submissions, and it took several studies to show that parts of CS should move to double-blind submission to hide author identity so that underrepresented authors were not as disadvantaged. I don't think it matters that reviewers can see the submission order. – Bill Barth Apr 29 '16 at 12:35
  • 1
    As a last minute submitter with an excellent track record of having submissions accepted I have to admit it never occurred to me that there was any advantage to submitting early. – Marianne013 Apr 29 '16 at 21:46
  • My personal theory is that submitting late is beneficial for getting accepted at EasyChair conferences: a low submission number correlates pretty well with finishing work early, which in turn correlates well with having been rejected at another conference before, so this submitting early could be an indicator for sub-stellar work and some reviewers start the work with a bias in mind. Obviously, I have no data to back this up. – DCTLib Jun 14 '16 at 15:57
  • 1
    @DCTLib You'd probably be prejudiced against my own papers, then, which often have a low number due simply to preferring to get things wrapped up promptly so I can deal with the next thing instead. – jakebeal Jun 14 '16 at 16:11
2

For instance, is it likely that an early submission will be picked up by a member of the program committee, rather than being handed over to a subreferee? When I'm going through a list, I do have a tendency to be more concentrated on the first few elements.

PC members often bid to review the papers that are closed to their research area. Because these papers may be relevant to their research, it would also take less time to review, and it would be easier to review since they already have a lot of background.

AFAIK, even for very established researchers, only a small fractions of submissions are closed to their research area. So indexes is very unlikely to be a priority.

After review bidding, there are papers, from the tittle and abstract, that everybody want to review, e.g.:

There are also papers that sound so scary/weird that nobody want to review. Therefore, the conference chairs have to do the tedious and labor-intensive task of assigning papers to PC members. This process eliminates any tiny influence of indexes that may have.

0

AFAIK the following is a general overview of how the review process works for most conferences:

  1. Submissions are collected until the relevant deadline.
  2. The submitted papers are assigned to reviewers.
  3. Reviewers review the papers and hand in their scores.
  4. Papers are accepted or rejected based on the scores and the reviews.

I have never heard of a process where papers are assigned to reviewers on the fly, before all of them are received. For many reasons (managing reviewer workload, reviewer/subject match, fairness...) papers are assigned to reviewers after the submission period is over and all papers are handed in. Thus, it does not matter when a paper is received, as long as the submission is made within the relevant period

Introducing the submission date (or a similar variable) as a factor into submission evaluation process would mean extra work for the organizers, without contributing value to the end result.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.