What is a strategy to do theoretical research?

Question

Thanks to some great SE answers on a previous question I posted, I realized that even though I don't have an RA at the moment (I am a first year PhD student), I must start my research ASAP. I am convinced that this is great advice and want to follow it.

But the concept of 'doing research' is still not perfectly clear to me. I am interested in convex optimization, that's what I intend to do my PhD in. So as of now, for my thesis, I have this vague idea in my mind that I'd solve a problem by maybe coming up with a cool algorithm and give some nice proofs using convex analysis.

But exactly how to start is a mystery. Here's what I think I should do: read a paper that looks interesting to me (this could be because of the abstract, or maybe it's a paper that has suddenly got very popular, or it could simply be based on the author's reputation, etc), thoroughly understand their results by working out proofs of all their statements, and then try to see what happens in slightly different cases than what they mention. Maybe relax some assumptions they make; or go the other direction, add a constraint and see if the algorithm can be improved because of this constraint, etc.

Is this how it's done? I feel so lost right now. When I try doing this, I have so far (in six months) not been able to really think of anything new. Even the existing proofs that I get are so complex, I can't think of any way to come up with something new. Can someone please give me some tips on how one gets started in theoretical research - how to come up with a problem, and how to proceed from there?

EDIT_ Adding more information: I don't have an adviser yet, and this is the post I am referring to in the first paragraph.

Answer 1

In mathematics, a paper with 20 pages of proofs does not have twenty pages of original ideas. It might have a page worth, maybe. The rest are combinations of 2-6 previously known ideas, often with little or no acknowledgements of their first appearance [footnote 1].

To beginning graduate students, I strongly suggest to read much older papers. They are usually much shorter, much simpler, and actually allow to learn ideas in their simplest. In my branch of mathematics, that means papers that are at least 30 years old. General useful mathematical background papers go at least 60-100 years.

A good way of doing research is just to be curious --- after all this is what got you into the position you are now, right? Whenever you are curious about something, try figuring it out, look for some way of doing it, search references, read whatever looks interesting, swear at whatever does not. Keep journal with your questions, observations, dead-ends, ideas and swearing. Occasionally, when there is something particularly nice, add those 5, 10 or 15 pages of formal expository and technical scaffolding that is needed to convey your message to others [footnote 2] and share with the world. That is research.

[1] Many find it embarrassing, awkward and out-of-place to write ".... by Cauchy-Schwarz inequality. (This way of applying Cauchy-Schwarz inequality I learned from [ref] which is completely unrelated to the current topic). Next we can bound (5) by..."

[2] After spending much time thinking about X, you will find that you need to spend much time to explain the basics of X to anyone who has not spent the same time thinking about X. That is why you need all that scaffolding.

Answer 2

What you describe (look at paper, understand it in minute detail, look at next paper) is a depth-first search. This is probably a bad idea (oblig. xkcd).

It's very common at the beginning of a PhD to do a proper literature study. This means writing a report (maybe just for yourself and your supervisor) that summarises everything the human race knows about convex optimisation. Of course, this will not include detailed proofs of everything. The point is to get an overview of the field. If you're lucky, someone has recently written a review paper in your (sub)field, which helps a lot.

When you have an overview, you will (hopefully) at least know one very important thing: which problems have been solved, and which problems are still unsolved, or solved in a poor way? Then comes the hard work: trying to come closer to solving the unsolved problems in your chosen sub-sub-field.

Answer 3

I'm a Ph.D student who seems to be doing well and from my perspective, I think it came really naturally by finding an interesting problem, and just "doing it". I started in a Math Bio program and switched into the Math Ph.D, but what I made a bunch of connections with Bio students and found some interesting problems in zebrafish that I wanted to model. However, there weren't the right mathematical tools to computationally simulate it at the error I wanted (stochastic dynamics), so I started doing what made sense, went through the literature to find everything out about the area, and started making algorithms to solve the problem.

Convex optimization is a field a lot like stochastic dynamics where you can find real problems that need to be solved. Go find one that's interesting: cancer biology, mathematical finance, etc. and try to solve it. Maybe you get a free "applied" paper because existing methods work. Most likely, it hasn't been done because it's mathematically/computational hard, and now you have a problem that is both mathematically and scientifically interesting. With a concrete idea of what you need, it will be much easier to move forward. Even if you don't want to do applied work, you will find "pure" theoretical work that needs to be done that is also very applicable.

Also, I second Semi's suggestion to start writing. I started by thinking that reading counted as doing work. How many times have you read a proof and thought you understood it, but couldn't reproduce it with the book closed? I took my advisers advice: reading doesn't count as work, only count writing time. Read everything and compose your own notes, or basically your own book on the subject. Prove extra generalizations / interesting side stories as you think of them (some of these will later go into a paper!). Sooner than later it will become the best source of information that you know of on the subject.