Doing research on my own — Is it possible? What alternatives are available?

Question

Currently, I am working in a field of applied computer science. I’ve been reasonably successful in industry after finishing my PhD a couple years ago in this same applied field.

For a bunch of personal reasons — financial security, politics, sunk time, fear of failure — I decided to finish my PhD in the field I did, despite general unhappiness with it. The truth, though, is that I would really like to do something theoretical. For the last couple years, I set my sights on a particular subfield unrelated to my existing research, and I’ve spent that time reading about it a lot. Probably too much, as the time I spend reading papers and textbooks has slowed down the research I’m supposed to be doing.

I’d really love to publish a paper in this other field. The problem is, I have no idea how to pick a topic that’s both interesting and tractable. Everything I read suggests that you really can’t do highly theoretical work in a vacuum, and that definitely feels true from where I’m standing.

This is made worse by the fact that I didn’t even really master that skill for my current (much easier) field during graduate school. My advisor generally gave me awful, low-impact topics; all of my successes came in industry where problems were mostly picked for me. What little problem-selecting skills I have aren’t sufficient for the much denser research landscape of theoretical CS. Even in the area I’ve honed in on, the list of topics is vast and many solutions seem to come from synthesis with outside ideas.

And yet, with even the most minimal mentorship, I firmly believe I’m capable of producing at least some interesting research. I took several highly theoretical CS courses in graduate school and did just fine - it’s hardly proof I know but that must count for something.

I’ve even emailed a handful of professors about this, mostly stating what I’ve studied and asking for guidance in problem selection. The ones that have gotten back to me at all were helpful and encouraging but were naturally too busy to really give me anything concrete. Of course, I made it clear that I didn't expect any real guidance beyond a short initial meeting, but I can't blame them for not being interesting.

Is there any hope for me, or did I miss my chance by not going for it during graduate school? I know I should have taken my chance then; am I now locked out?

If not, what are the steps I can take? I’ve started scouring for topics by looking for mentions of open problems in papers, should I just keep looking? I feel like the problems I find all seem too imposing, and lack of confidence keeps me from giving any of them a real chance. Am I just wasting my time?

Answer 1

It sounds like you are spending too much time agonising over finding the perfect problem, rather than just getting stuck in and trying to work out something of interest to you. You've said that you've been reading a lot about the topic you want to pursue --- usually that would raise some questions you have that go beyond what you've read, and that could give you potential topics for papers. Don't worry too much about tractability a priori unless you have good reason to do so. Just pick a problem that is interesting to you and try to solve it --- if it is not tractable you will find that out in the course of trying to solve it, and then you can pivot to a smaller/related contribution to the problem.

The physicist Richard Feynman made the point that he started doing his best physics work when he started playing around with small toy problems that were interesting to him but with no wider importance. I'd recommend you start playing around trying to solve some problems in this topic that are of interest to you and see if this leads to some novel research. You have been trained up to PhD level, so academic research is certainly possible for you.