3

What would I learn as a Ph.D Student in Statistics that I wouldn't learn and vise-versa in Machine Learning Ph.D program?

Improve this question
2
  • 3
    Note it is vice versa...
    – Solar Mike
    Aug 1, 2019 at 20:38
  • 1
    From a purely business perspective, statistics is what you do. Machine Learning is what you tell your client you're doing
    – David
    Aug 2, 2019 at 9:29

5 Answers 5

Reset to default
3

What is called "machine learning" is essentially just statistical prediction, but coming at it from more of a computer-science background, with a greater focus on non-parametric algorithms, divergence measures, and big data problems where computation issues become a concern. The field includes many standard statistical techniques, but many of the algorithms in use are complex non-parametric algorithms where the full statistical properties of the algorithms are not yet known. The field seems to have arisen as a consequence of computer scientists entering into the field of statistical prediction, and approaching that field in a different way to the traditional statistical field. There is less emphasis on foundational theory (though that is changing) and more emphasis on algorithm development and computational issues. There is a greater emphasis on non-parametric models and simulation, with theoretical underpinnings framed in terms of properties of divergence measures.

Since machine learning is essentially replicating one subject within statistical analysis, there is a huge cross-over in material between these fields, and anything in either field can be regarded as also useful to the other. Indeed, arguably machine learning is not even a new field at all; just a new approach to an old field, with slightly different points of emphasis. In any case, machine learning is likely to focus less on foundational issues in probability theory and statistics, and more on computation and programming of a particular 'toolkit' of predictive methods (e.g., sparse models, neural networks, classification algorithms, random forests, etc.). Theoretical underpinnings of machine learning are presently in development, and tend to focus on the mathematics of divergence measures, with convergence often framed in terms of 'order' arguments framed in classical computational terms.

In terms of what you are likely to learn in each PhD program, that is largely self-driven, so there is no reason you cannot learn material from one field while formally enrolled in the other. Indeed, regardless of which you pick, I would strongly recommend that you try to learn as much as possible about both. If you become a statistician (like me) then it is useful to have familiarity with the material in machine learning, and if you go into formal computer science and machine learning, it is useful to have familiarity with probability theory and statistical science. The mathematical foundations in a statistics PhD will focus more on measure theory and probability theory, and the applied work will focus more on traditional parametric and nonparametric models, whereas in machine learning the mathematical foundations focus more on broader divergence measures (e.g., Bregman divergence), and the applied work focuses on certain types of nonparametric methods that are on the fringes of traditional statistics.

Improve this answer
4

Machine learning is a quickly developing branch of technology, based on some basic statistics but also including elements of computer science, high performance computing, etc.

Unfortunately, ML/AI is currently a buzzword and many departments want to have a course on it assuming it will attract students and bring money. This means that when it comes to taught courses, there is no consensus yet what should definitely be included and how the course can be structured. There are examples of ML courses drawing significantly from the classical foundations of Statistics, courses more focused on use of software, more vocal courses with significant discussion of ethics / legal issues around artificial intelligence. Similarly, a PhD project in ML can be either highly theoretical (e.g. proving a convergence or stability of training), or highly practical (e.g. building 100s of NNs until one of them does better than others for some reason), or whatever else.

Statistics is a more established science, which has many applications. Depending on the area of applications, there may be some variations (e.g. Medical Stats may be slightly different from Statistical Physics), but the core methods are still the same, largely based on probability theory, calculus and linear algebra. I would not expect a PhD thesis in Stats to avoid maths completely (something which I can not say about ML).

Improve this answer
3

Statistics is a tool with many uses. It is, at its core, unrelated to machine learning, so, in a statistics degree you might never hear anything about machine learning.

On the other hand, many techniques of machine learning depend on a fairly deep knowledge of statistics, so you'd better come to such a degree program in machine learning already knowing at least the basics of statistics. In fact, the basics may not be enough, depending on the faculty of the program and their preferred toolset.

Improve this answer
0
0

Both are about getting answers from data, both can lead to good jobs (if you are good at them), and makes a lot of sense to take courses from both fields.

Statistics is more about theoretic foundation (in Probability theory), and methods that are consistent with that theory. Expect math-heavy coursework and dissertation. Likely branched out from Math department, more suited to academic types.

Machine Learning is more of a bag of tools, without much theory as to why some of them work. Expect homeworks and dissertation that focus on programming. Probably branched out from Computer Science; more suited to industry types.

If you are considering specific programs, look at their "placements" page, i.e. what kind of jobs their recently graduated Ph.D's get.

Improve this answer
3
  • 1
    what is math? proofs all day?
    – user96933
    Aug 1, 2019 at 20:10
  • 4
    I'd push back on ML being a bag of tools, especially at PhD level. You'd certainly need an understanding beyond just using high-level apis with little thought about what's going on. Recreational/enterprise use of ML is very different to a PhD.
    – QuantumChris
    Aug 1, 2019 at 21:02
  • yes, statistics will involve more proofs, like consistency of your estimation method. ML will involve more programming.
    – Bald Bear
    Aug 2, 2019 at 2:50
0

Doesn't it rather depend on what is the precise topic of your research? I am currently researching a subject as a member of my university's school of mathematics and statistics, but, for my subject, I need to know quite a bit about ML, and, as it happens, fluid dynamics.

Surely it is a bad idea to tie yourself into a narrow discipline prematurely.

I am in the UK, so coursework generally is not a feature of a PhD programme.

Improve this answer