How is an experimental particle physicist evaluated?

Question

As an undergraduate student majoring in physics, I intend to apply for a Ph.D. in experimental particle physics. I have known that working as physicists in this field is kind of different, as they are always affiliated to a collaboration (like CMS in LHC) and may appear as authors in loads of related papers, resulting in high citations. Therefore it seems traditional evaluations of a researcher (focusing on the citations or h-index) do not work when assessing an excellent and famous experimental particle physicist.

Thus I am asking:

• How are particle researchers assessed in reality, in the case when a Ph.D. applier is searching for and comparing professors in the similar field, or when a researcher is going to find a position as an assistant professor? Does the specific role of a scientist in the collaboration (say, the group leader) matter in this evaluation procedure?

• How does university ranking play into this? What are the probable differences between professors in particle physics who work at a higher-ranked institution (ranked by physics major) and those who work at a lower-ranked one – given that they are both affiliated to some big collaboration. This question comes to me since it seems one cannot distinguish a famous particle physicist by the conventional method which is performed by university ranking.

Answer 1

HEP PhD here.

What matters for a permanent position?

• By far the most important factor is the "physics." You need to have innovative ideas about how to discover particles. On the small experiments, this can mean new approaches for detectors, analysis, etc.; on the larger experiments, this usually means new analysis techniques.
• Another important factor is "leadership" -- you need to attend lots of meetings and make substantial contributions at all of them.
• Though it's understandable that universities want to hire the highest-impact people possible, this does lead to a slightly toxic environment. There are endless meetings where everyone argues over factors of epsilon, and everyone wants to be involved in dozens of things but not put time into any of them. Skills, teaching, and knowledge of other fields/subfields don't count at all.

How is this measured?

• The most important way is the letters of recommendation. You will need 3 for post-docs and up to a dozen for a faculty position.
• "Official" positions such as analysis lead, sub-convenor, etc. help a lot.
• Awards and grants also help, though there are not many of these for grad students.
• As you say, publications don't count at all (at least directly), since there's no way to tell who really worked on what, everyone's name goes on everything (this is seriously different than most other fields, and introduces serious drawbacks).

Finally, let me point out that here are ~10 US HEP faculty jobs per year and hundreds of HEP PhDs produced each year (and the lack of new particles means that hiring is unlikely to accelerate). So, there is some element of randomness.

As for university ranking: more prestigious universities get more qualified grad students (or even post-docs), which tends to make their groups more productive. But at the faculty level, people tend to go where the jobs are; there will usually not be meaningful differences in the qualifications of faculty at the #20 school vs. faculty at the #80 school.

Answer 2

I happened to be a member of my college's executive committee when an experimental particle physicist was being considered for promotion to tenure. She was, indeed, a member of a huge collaboration, so a typical paper of hers had perhaps 100 co-authors. The papers provided no information about what, specifically, she had contributed (and we know better than to pay much attention to citation counts and impact factors). The information we needed, about the quality and quantity of her work, was provided by numerous letters from external reviewers. ("External" means not in our university.) Ordinarily, the college requires that most external letters come from senior researchers who are not co-authors of the candidate's papers, but an exception is allowed in areas, like experimental particle physics, where only co-authors really know about the candidate's work. My recollection (about 15 years old and therefore somewhat vague) is that the letters gave us very detailed information, not only about what the candidate had contributed but also about how efficiently she had done her part of the work.

Answer 3

H-index is a simple metric - and as it goes for simple metrics, they are mostly used by people who are not in the field, or funding agencies. As you say, h-index just does not work in particle physics, where h-index naturally grows over time, just because you are a member of a collaboration. To give you a feeling, the average PhD graduates with an h-index around 40. I met a young researcher that changed experiment twice in his PhD career and because of bizantine rules continued to sign papers from all three big collaborations. His h-index was around 70 before his graduation. One particle physics collaboration puts the main authors at the top of author list. Some had the main authors only listed on a publicly accessible webpage that explains the measurement. Most of the time, a great deal of effort is spent masking who the lead authors actually are. While experimental particle physics represents the extreme scenario, other fields share similar issues.

How are particle researchers assessed in reality, So, h-index is mostly meant for outsiders. People in your field - any field - judges by reputation. Reputation is built by being consistently successfull in leading original work, being a good disseminator, a respected leader, etc. How to measure things without looking at papers, you might ask?

1. The institution that awarded your PhD. Having a PhD at MIT or similar institution is the earliest sign of outstanding skills, and it is probably the one that sticks with you the most.
2. Your PhD supervisor. If you graduated with someone who is already famous in the field, and he/she highly admires you, that is the best walking letter of reccomendation you will ever have.
3. Talks/seminars. Great research work leads not only to highly cited papers, but to invitation to present your results at seminars at key institutions, or talks at key conferences. A hiring committee member will be able to map the (selected) papers you display on your CV to the talks you gave. While talks/seminars are important per se, it is probably the networking the follows that matters the most.
4. Breadth of research. In particle physics, that means working both in "hardware" - building/operating the detectors - and in "data analysis" - producing the measurements themselves. If you collaborate on theoretical papers, that's a plus. By the way, "hardware" and theoretical papers have "normal" authors list, i.e. list only the main authors.
5. Prizes/awards. Admirers will nominate you for prizes. You will feel you have a shot at winning awards, and will apply to all of them. Prizes and awards lead to more prizes and awards.
6. Reputation prompts invitations to lead scientific efforts, that produce higher returns than working solo. Leadership in turn prompts additional reputation boost.
7. Luck. As pointed out, the number of High Energy Physics opening is so limited, and the number of PhD awarded so high, that you are most likely to get invited to interviews if somebody in the hiring committee already had a chance to encounter your work directly, and/or was impressed by a public appearance of yours.

How does university ranking play into this? With this question you actually meant how to rank established professors. Established professors reputation works as for younger researcher - basically the points 1) to 6) above. You can replace Luck with Group size. Group size is basically a proxy of how a professor reputation gains her grants to hire postdocs, and aura to attract students.

As you can see, most of the above is actually pretty general, and holds for most field. What characterizes experimental high energy physics is the complete lack of interest in h-index (which would anyway just be a poor-man proxy for the points highlighted) and the extremely competitive scenario, that puts additional emphasis on visibility, networking, and a bit of luck.

Answer 4

Disclamer: I am a ex Expertimental Particle Physicist

In addition to the above answers...

Like any other social endeavor - and yes working in very large groups of highly intelligent people is a social skill rather than scientific; all the ills of the society at large also occur here. This includes all sorts of discriminations, politics, backstabbing, sleeping your way to top, plagiarism etc. It is just like corporate rat race but between very competent people who are motivated and have a whole lot to loose. Anything goes. Most of it is hard to prove.

You can present results and make a name only if you are allowed to do so. Every tiny career step will be hard.

If you keep your head down and learn and absorb everything this is the best real education you can have. Your are sharing room with Nobel prize winners and can see then in action. Hopefully you can learn their thought process. You can also be happy that at least for a short while you followed your dreams before life interrupts.