2

In my field, conference submissions are usually scored by all reviewers during the review process (for example using the following scores: 2 = accept, 1 = weak accept, -1 = weak reject, -2 = reject). Many conferences includes the review scores in their notification mails, which makes a lot of sense, as the scores add a relevant piece of information to reviews. They communicate the actual assessment of the paper, even if the review's tone sounds more positive or negative than the reviewer desired (which might have all kinds of reasons, be it culture- or personality-specific). They also make the decision on a paper more comprehensible to its authors.

However, some conferences do not include the review scores in the notification. What are reasons for not including them?

0

4 Answers 4

4

Conferences do not have a uniform system for how they select papers. Giving papers a score is a very odd concept in my field, and I am not aware of its ever having been done in any of the conferences at which I have presented or for which I have submitted proposals. Alternatively, it may be that scores were given but kept confidential.

0
3

I have been PC-chair of a recent conference run using Easychair, and I decided to not include the scores in the notifications for the authors.

First of all, why the reasoning that the score could help to interpret the reports makes sense to me - but wasn't really supported by the evidence for us. Whenever we perceived a discrepancy between score and report, we tried to clarify - and it was essentially always the score which was off. There doesn't seem to be a clear consensus of what a +2 or -1 mean. [We provided guidelines to the referees, but that didn't solve the issue.]

The role of the pure scores in our decision making process was extremely limited. This may be different for a conference with three or four digit numbers of submissions, but for <100 submissions, an individual discussion and comparison is quite doable. But this means that we did not adjust misleading scores for the selection progress.

When it came to sending the notifications, I felt that the scores weren't needed for the authors to understand their reports, and in a few cases could rather lead to confusion.

1

One possible reason for keeping scores confidential might be to avoid people who got a "weak accept" feeling that they don't belong there, and to avoid people who got a "strong accept" generating an unhelpful level of arrogance.

I don't work in a field where scoring is a thing, so I don't know if this is the reason - but ultimately, the only people who can tell you the reason for a given conference are probably the organising committee of that conference.

0
1

One possible reason is to "soften the blow" to authors of papers who only received very weak scores. Receiving very weak scores can be discouraging to authors, who might even avoid the conference in the future if they feel that their work has not been treated fairly. By giving them the reviews without scores, the authors are prompted to focus on the textual feedback and improvement suggestions instead of the severity of the rejection.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .