If someone doesn't get into a graduate program/postdoc, or a grant, etc, how are they supposed to know why?

Question

So that they can do better next time. There's usually no transparency or explanation of why they were rejected.

Answer 1

You generally won't know. That's frustrating because you don't know what to improve. All you know from rejection is that your application “wasn't good enough compared with the other applicants”.

The usual way to figure out what you need to improve is to give your application documents to someone who has been on the selection committee a few times and can give you feedback on what they see as your deficiencies.

Answer 2

Ten years ago, towards the end of my first year as an Assistant Professor, I applied for an NSF (National Science Foundation, the main US grant funding agency for natural and social (but not health-related) sciences) grant. It was rejected. I got some comments, but they weren't really all that helpful. As another agency funded a similar grant (albeit at a much lower funding level), I did not apply for a grant the next year. This gave the NSF a perfect opportunity to give me feedback; they invited me to sit on a panel evaluating grants.

What I learned from sitting on that panel is that I simply don't have good enough ideas and haven't done sufficiently groundbreaking research to get funded given the competition and the funding rates, even if I could propose research that would be, in an ideal world, worth funding. I have never applied for another NSF grant since.

Sometimes the only feedback is that you're doing all the things you should and can, but you're just a third-division footballer trying out for a first-division team.

Answer 3

I disagree slightly with Wolfgang Bangerth's answer that a rejection always means your application "wasn't good enough." The question also assumes that it's theoretically possible to have transparent rankings--and improve them.

Merit certainly matters, but decisions are often subjective and not under your complete control.

Hiring

For example, in many departments, students are selected by an admissions committee. These committees certainly consider recommendations, grades, etc but they also try to select students that will fit in well with department, balance the cohort across subfields, etc. If your (prospective) advisors doesn't have money/room for more student--or is just on the outs in departmental politics--there's not much you can do. Consistent with this, I think many people find that their "success rate" for grad school apps is only somewhat correlated with the ranking of the department. While these matter less for individual positions, interpersonal factors might matter: someone may fit in well with one group, but not get along with another. Having or being eligible for outside funding can certainly tip a decision. Finally, you may just get unlucky and face particularly tough competition.

Feedback isn't usually offered for PhD and postdoc applications, but if you've spoken to an individual prof, you can certainly ask (I would not bother if it was at the committee stage). People do sometimes get useful feedback. The other common option is to ask a mentor to look over your application. If you can, I would ask several people and focus on common themes.

Grants

Grants are a bit different in that you often get some kind of feedback or summary statement, but these can be tricky to interpret. A sufficiently motivated reviewer can find a flaw in almost any proposal: it's impossible to forestall every possible criticism in the 6-12 page format. As such, you need to read between the lines to determine whether the reviewers are making a specific technical critique (that you can fix) or are merely justifying their overall lack of enthusiasm for the proposal. Feedback from others can help, and you may be able to get some information from the funder's representative (e.g., the SRO at the NIH). Seeing other successful grants, either by asking colleagues or serving on review panels, is really helpful.

Still, the process is incredibly noisy. The primary reviewer may love your proposal or hate the entire subfield with a fiery passion. As with grad school apps, people have stories of a proposals that went from "not discussed" to top-ranked.

Answer 4

My answer is specific to Grants:

I don't fully agree with Alexander on classifying scientists into leagues and those of the premier league deserve funding. Like in football or in soccer, this categorisation is very subjective as the selection. This applies also to research papers, where borderline papers (which in my opinion represent the majority of submitted papers) need luck more than anything else to be accepted.

A few years ago, I was working with a professor with an outstanding profile and I used to write proposals for him and under his name. The quality of these proposals is bad and both the content and desired outcome are vague but most of them got funded (millions of €). The review comments are usually around:

The applicant seems to have all ingredients to make this proposal successful.

The outcomes of most of these projects were very modest.

I decided to write my own proposal after that, which took me one year to write and I wrote it more carefully than one would write a paper. I sent it to top experts in my area and they loved it in terms of writing style, content, motivation, planned outcome, etc. However, the proposal got rejected without any justification. I presumed that not my proposal which was not competitive but my profile, which is completely normal I think.

Answer 5

About 30 years ago, I sat on a grants advisory committee for the university's science faculty (physics, chemistry, math, computer science). We would not make the formal decision (that task lied with the board), but our recommendations were usually followed by the board. (Most of the board consisted of professors, but there were also two seats for ph.D students, one of them I occupied).

The major deciding factors on whether you are rewarded a grant? 1) How much money is available in this round, and 2) how many applications are there. Typically, we'd filter out applications which were below par (not many would be below par), then sort the rest. Where the cut-off point would be depended on the amount of money available (but that would be determined by the board).

As on how to sort applications? Lots of bickering and superficial things. People from the chemistry/physics department: "nah, he isn't good enough, he's never been a first author". Math/CS people: "that's because his name starts with a W, and in our field, we tend to sort authors by name" (Yeah, that discussion we had each and every meeting). People had the tendency to count number of publications (because that's easy), but that's just quantity, not quality. And number of publications would favour older applicants over younger ones. People would look far more into what an applicant has produced than look at the application itself.

And people on the committee would be far more favourable to applications from their own department than to applications from other departments. Let's just say that this was because it's easier to understand an application from your own field than a foreign field, and it has nothing to do with "the more money which flows to our department, the better".

Feedback was not given to the applicants. And I think they were better off.