For an interdisciplinary symposium: is it more fair to the participants to:

  • Average scores of submitted abstracts across different reviewers?
  • Estimate the overlap of the scientific domains between a participant and the reviewers, and weight different reviews accordingly? (which may introduce some bias)

(A detailed examination of reviewers is not possible. Reviewers are quite mixed, and come from different domains. Reviewers should be capable to judge most abstracts.)

  • 1
    To be honest, neither of the options you present sound particularly "fair". In my field (computer science), it's fairly common to normalize review scores by looking at the distribution of scores for the 30-45 submissions each program committee member is responsible for. Also, each reviewer is asked to rate their confidence/expertise in addition to rating the paper. – JeffE Aug 1 '16 at 15:18
  • But in any case, I hope you're not basing your acceptance decisions on scores but on the content of the reviews. – JeffE Aug 1 '16 at 15:19
  • it is indeed only an intermediate step – tsttst Aug 1 '16 at 15:19
  • @JeffE Just playing Devil's advocate, but presumably rejection and acceptance is an implicit scoring of the abstract as 0 = reject or 1 = accept. – Jeromy Anglim Aug 2 '16 at 2:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.