I am currently an undergrad junior majoring in physics and mathematics and will likely apply for a high energy experiment or observational cosmology PhD programme after graduation. I have heard quite a few times that many people who studied physics and mathematics in undergrad wished they had studied more computer science; especially if their graduate research is related to high energy experiment. Since CS skills are needed everywhere, even if I were to quit academia, I am trying to take some CS courses before I graduate.

Question: How much data analysis and what sort of analysis do you do in high energy experimental/observational cosmology programmes?

As of right now I am planning on taking a few courses on basic programming (data structures etc.), a course on statistical analysis for data science, and a course on machine learning.

  • 1
    Learn source control concepts and practices, how to develop and interact with core components such as relational and noSQL databases, continuous integration in practice, service-based architectures, etc., though all this can be done effectively outside of the classroom. If the opportunity to take a numerical analysis course presents itself, that's probably worthwhile. Similar for algorithms and data structures. Jul 23, 2021 at 19:59

2 Answers 2


I'll answer from the cosmology perspective, though probably a lot of this applies to hep-ex as well.

You should aim to have a good working knowledge of Python as that is the lingua franca in cosmology. Additionally, it's useful to be familiar with either Fortran or C, as some of the larger numerical cosmology codes such as CAMB and CLASS are written in those.

For observational cosmology specifically, you should learn how to handle large datasets and image files, especially FITS files. If you're going to be using existing databases of images, spectra etc, then learning SQL will also be useful.

Finally, the vast majority of statistical analysis in cosmology follows a Bayesian rather than frequentist framework, so understanding the basic concepts of that, as well as methods such as MCMC parameter inference will be useful.

Machine learning methods are becoming increasingly popular, though there is little true expertise in the field. Any knowledge and experience you have of using machine learning, especially classification and regression methods, will be a bonus.

If I were you, I would take a look at some recent papers in your specific field of interest to get an idea of the specific data analysis and computing methods used, as it will differ quite a lot from project to project even within cosmology. You can find relevant papers on the astro-ph.co section of arXiv.

  • Thanks for the clear answer! I gather that C and python are very important and data science courses that deal with Bayesian analysis are a good idea. But, is a machine learning course going to be useful? I would like to take if it's possible just because it's interesting.
    – Chandrahas
    Jul 25, 2021 at 5:27
  • @Chandrahas yes, machine learning would be a bonus if you can take a good course on it! I also forgot to mention that you should learn version control with git. That's quite simple and you can probably learn it on your own in a day or two (the basics, anyway). Jul 25, 2021 at 11:26

It will depend strongly on the specific tasks you are involved in. In most labs there will be the local guru who can get you started if you are nice to him. And on most large projects there will be several such gurus.

HEP experimental is going to, naturally, have a big hardware component. If you can repair, diagnose, build, or install some kind of electronic or computer hardware, then you may get to go to the lab. But consider how much time that will use that could have been used doing your thesis. If you can code the controller circuits for the hardware, that may be useful, with the same caveat. And in all such cases, you may be competing with laboratory tech staff.

But a PhD thesis in which you invented some new thing that helped out at some big lab would probably have a lot of appeal.

There will be tons of numerical analysis. Signal processing to understand and "decode" the output of a detector. For example, some experiments involve recording a huge volume of candidate events, then filtering it to find the specific category of events you are looking for. Database, AI filtering, and just plain-old number crunching. Say from a book such as Numerical Recipes. Note that there are versions of this book for several different computer languages, including C, C++, FORTRAN, maybe others. And you should treat this book as a "first intro" to the topics in it, looking for more advanced and powerful methods if you start working in the field. But it gives you lots of things in a form you can "get up the curve" on.

Another entire topic is Monte Carlo methods. This is a method of doing numerical experiments to try to predict the frequency of various events by throwing a bunch of random numbers at it. This would come at the data from the other end by attempting to predict what the detector would see in the case of a particular event. Maybe you can work out the "fingerprint" for a given event. Look for this combo and you know you have found the elusive anti-schmadron.

Another area is data visualization. When you are doing some horrendous large dataset coming out of some experiment, you want some way to present the information so people can comprehend it. Often you first. (Grin.) Maybe you want to learn apps that deal with such things such as MATLAB and related. Or maybe you want to learn to do 3-D visualization. There are several popular apps that deal with that, but I have not been involved in them.

If you are designing a detector (or some such hardware) you may want to learn 3-D CAD/CAM software. Again, there are several popular ones but I don't know them.

  • "... if your are nice to him or her" Jul 24, 2021 at 14:25
  • "HEP experimental is going to, naturally, have a big hardware component." This is wrong. Many particle experimentalists work exclusively on data analysis. Jul 24, 2021 at 20:10
  • The Numerical Recipes book is obsolete. Jul 24, 2021 at 20:10
  • @AnonymousPhysicist So sentimentalists don;t do hardware? OK.
    – puppetsock
    Jul 27, 2021 at 20:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .