Are best-paper awards for conferences rigged/pre-decided?

Question

Recently I had an acceptance from a very good conference in our field. Three out of four reviewers gave excellent comments, while one gave a poor comment. I was telling my friend that if I had had one more good/excellent review, I could be considered for the best-paper award.

He told me that usually best-paper awards are already fixed and given to the person of the organizer's choice i.e. if the organizers want to invite a distinguished professor, they award their paper etc. etc. So even if I had had an excellent review from the 4th reviewer, there was no chance of getting the award.

What is the backdoor working of best-paper awards - how are they selected?

Answer 1

It depends. I received best paper and other awards and have been (slightly) involved also in such a selection process. In each case, I have been to the conference for my first time, was not invited, and I am not very well known. However, it was possible to achieve awards.

In one conference, the organizers had two or three papers on their short-list. The final decision fell depending on the quality of paper presentation. That is absolutely OK in my opinion, I even prefer such an approach as I have seen good papers with really bad presentations getting awards.

At some other conference, I got some special award. At the end of the day, one of the committee members told me he had suggested me because he liked my paper. I did not know him before, thus, this was also not rigged.

When I was a session chair on my own, I was allowed to suggest papers for the best paper award. I had a look at the papers and afterwards at their presentations. I dropped some of them due to their bad presentation and finally recommended one or two (however, they were not awarded, I was just allowed to give my/one opinion).

However, I have seen quite the opposite, unfortunately. In one (not very well known) conference, a guy I met there told me that Ms. X will get some award AGAIN. This happens every year because an award makes it easier for her to get funding for next year. Obviously, she was well known there. And to my surprise, she indeed got awarded. Not a best-paper award, but still some nice certificate.

As far as I have seen, there are huge differences between conferences. I have never seen such bad behavior in highly-reputed ones. If something like that becomes public, they would lose a lot of reputation. Especially since this could mean, that also their peer-review is rigged.

Answer 2

I would find it very unusual and very disturbing if that happened.

In my experience (Computer Science and related fields), conference committees work hard with submissions to select the best paper. And they debate it, as different committee members will typically "champion" a paper that they think is worthy. Often this happens because the committee member is a specialist and recognizes that some paper makes a really significant contribution to some aspect within their speciality.

But I've never seen anything such as you suggest, nor any personal favoritism.

If this sort of thing really exists within a field, I think that it would be a good idea to raise the idea as an ethical issue with that community generally.

Answer 3

No. Conference best papers awards are not rigged. Often, associate or track chairs will make decisions based largely (but not entirely) on ratings from peer review. Sometimes, there are awards committees who may take recommendations from reviewers or chairs and make their own decisions. In conferences I've been involved in, awards have always gone to papers among those with the highest average scores.

Because paper quality can never been measured objectively, these decisions are subjective. The reflect the opinions of the award committee or chairs/organizers, and peer reviewers as to what the "best" work is. The process is no more rigged (and no less, I suppose) than the peer review process itself which is also inherently subjective.

There might be some very low status conferences where things are truly rigged but I'm sure these are rare and unlikely to be places you should be submitting anyway.

Answer 4

As someone who has organized award sessions and served on technical programming committee work across multiple disciplines, I would suggest that any conference that might consider such behavior as you describe would be one to avoid. If the organizers repeatedly use awards to entice speakers to the conference, that should be pretty clear from comparing speaker lists and prize lists, which are usually announced. So any sort of systematic bias should be a tip-off.

Answer 5

It is in the own interest of a good conference to select the best papers (according to some suitable, but ultimately subjective, criteria) for the best paper awards. The name of the awards is tied to the conference and if consistently good - in the sense of interesting, original, technically competent, or durable results - papers are selected, this improves the reputation of the conference. If the record is "iffy" or clearly rigged, the conference itself loses aplomb.

James Gleick in his book about Feynman makes this point that the Nobel Prize came to its reputation in particular because in its early phase it had the luck to be awarded to exceedingly important work. This, in turn, improved its own reputation.

Awards are always a two-way road and where not treated as such, there's not much point hoping for them, anyway.

Answer 6

None of the times I've seen a best paper awarded (in a different area to yours) have I considered the selection rigged, nor have I ever heard of a case where it was.

[I have, however, wondered in two cases about competence -- where the supposed 'best' paper had obvious errors - and in one case quite egregious ones - they'd have been shocking even if the paper had been hastily written just the previous evening by an undergraduate. I could say nothing, of course because my own papers had been in the running both times, and it would look like little more than sour grapes to bring them up after that selection. I'd have been quite happy if any number of other papers had won the prize, though. Even many years later I am left to wonder if anyone on the respective committees ever came to realize just how terrible the papers they thought 'best' actually were. It didn't say much for their standards!]

Even in the best circumstances, there can be the common sorts of cognitive biases, of course. Getting a well-known name added to your paper as a co-author can improve the selection committees view of it (especially if a few of them don't really understand it), but that's a different thing from being rigged or pre-decided.