38

Recently I had an acceptance from a very good conference in our field. Three out of four reviewers gave excellent comments, while one gave a poor comment. I was telling my friend that if I had had one more good/excellent review, I could be considered for the best-paper award.

He told me that usually best-paper awards are already fixed and given to the person of the organizer's choice i.e. if the organizers want to invite a distinguished professor, they award their paper etc. etc. So even if I had had an excellent review from the 4th reviewer, there was no chance of getting the award.

What is the backdoor working of best-paper awards - how are they selected?

  • 1
    Which discipline is this? Not all conferences work like TCS, which is what your example sounds like – Yemon Choi Jul 23 '18 at 3:43
  • 6
    Two comments: (1) I'm not sure any conference of any note rigs the "best paper" award. I can see this happening in conferences hosted by predatory companies, though. (2) Before believe rumours from your friend, I suggest asking him for evidence or doing some investigation yourself. For example, the Thomas Chalmers award given by the Cochrane Collaboration is pretty clear about the eligibility and assessment criteria. Good luck! – St. Inkbug Jul 23 '18 at 7:12
  • 28
    What IS true is that in some (many?) cases the winner(s) of the best paper award are informed that they have won the best paper award before the conference (I imagine for practical reasons, such as ensuring that they will be present when the best paper award is awarded). – user53923 Jul 23 '18 at 7:55
  • 2
    My own experience with the best-poster award was that it was not rigged. I came out of the blue (1st conference, 1st poster), I didn't know anyone and I got a poster award (about 5% of poster presentators earned it). I'm sure this is the same with the best-paper award. But of course it might depend on the conference. – thermomagnetic condensed boson Jul 23 '18 at 19:54
  • 3
    One more thought about this issue: Maybe the person you were talking to was confusing "best paper" with "invited talk" / "plenary talk" - you were descriping exactly the process how those talks are selected. – OBu Jul 24 '18 at 12:00
30

It depends. I received best paper and other awards and have been (slightly) involved also in such a selection process. In each case, I have been to the conference for my first time, was not invited, and I am not very well known. However, it was possible to achieve awards.

In one conference, the organizers had two or three papers on their short-list. The final decision fell depending on the quality of paper presentation. That is absolutely OK in my opinion, I even prefer such an approach as I have seen good papers with really bad presentations getting awards.

At some other conference, I got some special award. At the end of the day, one of the committee members told me he had suggested me because he liked my paper. I did not know him before, thus, this was also not rigged.

When I was a session chair on my own, I was allowed to suggest papers for the best paper award. I had a look at the papers and afterwards at their presentations. I dropped some of them due to their bad presentation and finally recommended one or two (however, they were not awarded, I was just allowed to give my/one opinion).

However, I have seen quite the opposite, unfortunately. In one (not very well known) conference, a guy I met there told me that Ms. X will get some award AGAIN. This happens every year because an award makes it easier for her to get funding for next year. Obviously, she was well known there. And to my surprise, she indeed got awarded. Not a best-paper award, but still some nice certificate.

As far as I have seen, there are huge differences between conferences. I have never seen such bad behavior in highly-reputed ones. If something like that becomes public, they would lose a lot of reputation. Especially since this could mean, that also their peer-review is rigged.

  • "In one (not very well known) conference, a guy I met there told me that Ms. X will get some award AGAIN. This happens every year because an award makes it easier for her to get funding for next year" - was this guy an insider in the award committee? Or is it possible that Ms. X just has a record of submitting good papers, and somebody has sour grapes? – Geoffrey Brent Jul 25 '18 at 6:56
  • @GeoffreyBrent Sometimes, to encourage a better-known individual to attend the conference/event and raise the prestige of the conference/event, they are told "if you attend, we will give you a special award for Y". This is typically (and I have certainly never seen it not be) a separate award to things like "best paper" - think of it as similar to "Lifetime Achievement Awards" or "Acadamy Honorary Awards" at the Oscars. So, it may be a case of "We know Ms. X only attends because she gets the 'Ms. X is famous' award every year" – Chronocidal Jul 25 '18 at 13:41
  • @Chronocidal yeah, I can believe that, but it sounds like a separate class of thing from what the OP was asking about, or what J-Kun seemed to be implying. – Geoffrey Brent Jul 25 '18 at 23:59
43

I would find it very unusual and very disturbing if that happened.

In my experience (Computer Science and related fields), conference committees work hard with submissions to select the best paper. And they debate it, as different committee members will typically "champion" a paper that they think is worthy. Often this happens because the committee member is a specialist and recognizes that some paper makes a really significant contribution to some aspect within their speciality.

But I've never seen anything such as you suggest, nor any personal favoritism.

If this sort of thing really exists within a field, I think that it would be a good idea to raise the idea as an ethical issue with that community generally.

  • 3
    It happens a lot in fact. The weight of the conferences in CS is not shared across fields. Sometimes someone gets an award for more political reasons than research ones. I have seen it happen, specially with more minor awards, such as best poster or best presentation (not best research, but presentation skills). – Ander Biguri Jul 24 '18 at 10:06
22

No. Conference best papers awards are not rigged. Often, associate or track chairs will make decisions based largely (but not entirely) on ratings from peer review. Sometimes, there are awards committees who may take recommendations from reviewers or chairs and make their own decisions. In conferences I've been involved in, awards have always gone to papers among those with the highest average scores.

Because paper quality can never been measured objectively, these decisions are subjective. The reflect the opinions of the award committee or chairs/organizers, and peer reviewers as to what the "best" work is. The process is no more rigged (and no less, I suppose) than the peer review process itself which is also inherently subjective.

There might be some very low status conferences where things are truly rigged but I'm sure these are rare and unlikely to be places you should be submitting anyway.

13

As someone who has organized award sessions and served on technical programming committee work across multiple disciplines, I would suggest that any conference that might consider such behavior as you describe would be one to avoid. If the organizers repeatedly use awards to entice speakers to the conference, that should be pretty clear from comparing speaker lists and prize lists, which are usually announced. So any sort of systematic bias should be a tip-off.

5

It is in the own interest of a good conference to select the best papers (according to some suitable, but ultimately subjective, criteria) for the best paper awards. The name of the awards is tied to the conference and if consistently good - in the sense of interesting, original, technically competent, or durable results - papers are selected, this improves the reputation of the conference. If the record is "iffy" or clearly rigged, the conference itself loses aplomb.

James Gleick in his book about Feynman makes this point that the Nobel Prize came to its reputation in particular because in its early phase it had the luck to be awarded to exceedingly important work. This, in turn, improved its own reputation.

Awards are always a two-way road and where not treated as such, there's not much point hoping for them, anyway.

1

None of the times I've seen a best paper awarded (in a different area to yours) have I considered the selection rigged, nor have I ever heard of a case where it was.

[I have, however, wondered in two cases about competence -- where the supposed 'best' paper had obvious errors - and in one case quite egregious ones - they'd have been shocking even if the paper had been hastily written just the previous evening by an undergraduate. I could say nothing, of course because my own papers had been in the running both times, and it would look like little more than sour grapes to bring them up after that selection. I'd have been quite happy if any number of other papers had won the prize, though. Even many years later I am left to wonder if anyone on the respective committees ever came to realize just how terrible the papers they thought 'best' actually were. It didn't say much for their standards!]

Even in the best circumstances, there can be the common sorts of cognitive biases, of course. Getting a well-known name added to your paper as a co-author can improve the selection committees view of it (especially if a few of them don't really understand it), but that's a different thing from being rigged or pre-decided.

protected by Alexandros Jul 24 '18 at 19:07

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.