I will be serving in my very first NSF Panel as a panelist and reviewer. While I am excited, I am also anxious to do a good job... Please provide any words of wisdom to be successful in this endeavor. Also please suggest if there are any Rubric that you aware with respect to reviewing NSF proposals... If so could you share / provide pointers to it...

  • 3
    I'm suggesting this question be closed and deleted. First, I don't know anonymous one really is here, and identities of panelists should not be known, especially before panels make recommendations. Second, there is every possibility an answer will be biased, not necessarily intentionally, towards the answerer's own proposals. More generally, the NSF wants to know what you, someone expert enough and neutral enough to be invited as a panelist, think makes good science in your area of research, not what others who may be less expert or in a different area think. Make up your own rubric. Nov 17 '15 at 16:39
  • 6
    ... I am not a PO, but I am pretty sure the NSF wants every panelist to have a slightly different rubric in mind and for the panel to argue it out in front of the PO. Hence having a good answer publicly available is actually bad here. Nov 17 '15 at 16:44
  • The NSF typically provides an overview in broad strokes of criteria used in reviewing proposals. While you are expected to include your own expertise, their guidance will ensure that reviewers are at least prioritizing the same criteria.
    – Teusz
    Nov 24 '15 at 8:00
  • First, you should follow whatever instructions the program director gives you. Beyond that, you should carefully read the proposals and keep notes of your opinions while reading them. Reviewers will have different priorities, for example about a proposal that contains very creative ideas but is sloppily written, or about a scientist with a strong track record in one field moving to a rather different topic. That's where the panel meeting gets interesting (for you) and useful for the NSF. Nov 24 '15 at 10:14
  • 2
    @ScottSeidman identities of NSF panelists are confidential, including after the panel took place. You are allowed to say in your CV that you were on an NSF panel, but not which one.
    – Dan Romik
    Nov 24 '15 at 17:18

I don't know much about the NSF process, but can share my thoughts on the NIH process, and maybe others can comment on how this applies. I think some of this is directly applicable.

Remember, you're reviewing grants and not papers. You have reached the state of your career where your actions can DIRECTLY IMPACT somebody else's career, and fairly immediately. It could conceivably even end some careers. Do not take this responsibility lightly.

The biggest issue that I see come up is "New Reviewer Syndrome", where a reviewer feels compelled to make him or herself heard in a room full of people, even though the criticisms offered are nickel and dime stuff. You can't offer a list of five trivial matters and then recommend a high priority score. It doesn't work that way. If you don't have substantive criticism, best to keep quiet.

That said, if you have substantive criticism that you believe can turn the grant from unfunded into one that's funded on the next round, SAY IT AND SAY IT LOUDLY. Communicate it in a way that the applicant can hear it and understand it, and know that this is the issue separating him or her from a recommendation for funding. Don't make the applicant guess about how important you consider this issue. Failure of the applicant to understand what the study section wants will result in extra rounds, lost time, negative career impact, etc. If the person running the section sends you some synopsis of the review, read it and make sure that it conveys the feeling in the room during discussion. If you are asked to write such a synopsis, take that job VERY SERIOUSLY.

Another issue I have seen repeatedly is that its very tempting to score the grant that you wish the applicant wrote, and not the grant the applicant wrote. Don't judge the grant based on how you would do that research, and remember that there's more than one way to skin a cat.

Your institute will provide you with some good guidelines, but here's a framework to start with:

  1. What question is the applicant asking of the proposed study?
  2. Is that question worth getting an answer to?
  3. Will the proposed study answer that question?
  4. Is the applicant/applicant's environment capable of successfully carrying out that research?

When you're outside of the review room, KEEP YOUR MOUTH SHUT. Never violate the confidentiality of the process. I've had people come up to me in conferences and start telling me about what was in my applications, even though they were not on the study section. This put me in a situation where I had to moan to my program officer that they had a leak, and they should shut it.

Listen to your peers in the room. Some have been doing this a lot longer than you, and some are in a better position to judge a particular application than you are. Decide if their input sways your opinion, and act accordingly.

Don't get provincial about defending your area of research to the study section. Your colleagues in the room will be doing enough of this. Research that is what you do isn't by definition good research, and research that is not what you do is not by definition bad research.

Lastly, whatever serves at NSF as the equivalent of an NIH review officer is there to help you. Read and understand whatever review guidelines you are sent by the powers that be. At NIH, many instructions go toward trying to get the study sections to use the full range of scores. If you can do this, this makes life much easier for the review group. If you can't, you really start getting the feeling that the pay cutoff is quite arbitrary.

  • 2
    I have been a NSF panelist/reviewer many times, and I have to say that @ScottSeidman's advice from the NIH perspective is absolutely spot-on. Nov 25 '15 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.