2

I'm a 4th year PhD student (in 5 years program) and recently I have been invited by a conference organizers to become a member of the conference technical program committee (TPC) for an international conference in Bahrain. I have discussed with my PhD adviser whether to accept such invitation and he mentioned that it is good to participate in the conference activities.

So, what things I need to consider while reviewing papers since this is my first TPC task, any tips?

I found some question here but mine is a bit different.

3

As a reviewer, you have a dual role: you are both, a gatekeeper responsible for ensuring the quality of the programme, and a mentor to the authors for improving their own work.

  • As a gatekeeper, your task is to fairly evaluate whether a paper fulfills the standards of the community and this specific conference, whether the paper is methodologically sound, and (notoriously) whether the paper is sufficiently interesting to the community. The last part tends to be the most subjective, and my personal recommendation here is to not put too much stock on this item. By and large, we as a scientific community tend to overestimate our ability to predict what research will have an impact. Further, don't be too strict with regards to methodological soundness - my impression is that particularly inexperienced researchers have a tendency to be overly critical of other people's work. Finally, be careful not to recommend rejecting a paper simply because it is not the kind of research you would have done, or because the authors have chosen a different method than the one that you would use when attacking their problem.
  • As a mentor, you want your review to be both, honest and helpful. Don't sugarcoat your review, but make sure to highlight positive as well as negative aspects of the manuscript. If you criticise, do so with concrete, actionable recommendations. Don't just write that "approach A is bad", but suggest what else the authors could do that would be more convincing. Don't write that the authors have not taken important related work into account, but list what exactly is missing and why this matters. Always, always make sure that your review is professional and friendly - you may be tempted to give some authors some "tough love" and set them straight. Avoid this temptation at all costs.

Some more recommendations:

  • Don't write a super-short, high-level review. As you are surely aware, as an author it is very disappointing to receive a 5-lines review for an article that you spent a lot of time on. However, it is ok if the length and detail corresponds to the quality of the paper - for a very questionable submission that is clearly below the bar, you don't need to write pages and pages of explanations if one or two paragraphs are sufficient. For instance, if you are convinced that the method the authors use is fundamentally incorrect, there is no point in detailedly commenting the entire manuscript.
  • Make sure that the crucial points of your review stand out. If you write a detailed review (as you should), highlight what the most important comments are, so that they are not drowned out by smaller, detailed comments. If you recommend rejecting a paper, I suggest clearly differentiating issues led to you rejecting the paper and things that you think are either easy to fix or less crucial problems.
  • Give the authors the benefit of doubt. Don't assume that everything that isn't described in sufficient detail is necessarily terrible, or that every mistake in the paper has been planted with malicious intend.
  • Relatedly, make sure that you don't read anything into the paper that isn't actually described. Our mind tends to fill in the blanks, and sometimes all it takes is a few keywords to fill your mind with an idea of what you think the authors have done. Make sure that you are evaluating what the authors actually describe, and not what your mind imagined after the abstract.
  • You have a duty to try to understand the work, but if you fail to understand what the authors actually did even after honestly trying to follow, it is on the authors. Ultimately, you are a prototype reader for the paper - if you fail to follow the manuscript despite presumably investing more time and goodwill than the average reader, the paper may need a better presentation.

Overall, remember your own experiences as an author - which reviews did you find helpful? Write reviews that you yourself would appreciate, especially if you recommend rejecting the paper.

1
  1. I would assume they may need other things from you than just reviewing papers (moderating a session or chasing people down or whatever). Just pitch in and help.

  2. For the reviews, I advise not to be as tough as you would be for an APS journal. It's not the expectation that the hurdle is the same. And also a lot of people do conference papers that are little vignettes or mishashes of previous work. Yeah, the advice says not to do this...but it is incredibly customary. [You want practical tips, I give you this.]

  3. If you can improve a few papers that are hard to read/comprehend that is all for the better. I would even see this as the main work versus scientific quality control. Even here, your objective is to get the papers over the hump (conference really wants the papers for their proceedings). It's very different from PRL getting many times more papers than they accept. So be supportive and helpful and if you have to reject something, try to make it clear and easy for author to revise and get accepted after revision. (It would be a very rare paper that I would reject after second try even if still issues with it.)

  4. For clear easy papers give a little feedback and accept without revision. There will be a lot of papers and you need to keep things moving. I wouldn't even feel the obligation to write a long report (as another person advised).

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.