I guess, this question might be not only of my personal interest: I'm a first year PhD CS student, during my BSc and MSc, all research projects I did were purely practical - we hypothesize that certain way of computing or approximating something is "good" in some sense or another. Last year I was mostly doing deep learning that reduces to "if we wire network like this, those interesting properties might appear, here're some clues on possible reasons". But I used to (and still do) like doing math, learning new concepts and figuring out proofs, but:
a) I have no idea of how to come up with such a research project (closest thing to "theory research" I did was proving convergence bounds)
b) professor I am (have been for a while) working with now (and who partially funds my RA) is not very inspired by my intent to do research in theory (with no guarantees that anything would work out) instead of performing real-world publishable experiments (that also rarely lead to successes, but are at least more "measurable").
More specifically, I have been doing machine learning last few years, and I know that there exist a bunch of cool areas to explore and I took (and I liked) courses on statistical learning theory, optimization, Bayesian inference and other more rigorous things. But I just don't know how to start "doing research" there.