32

I was reading the transcripts of a seminar by Dr. Richard W. Hamming. In this, Hamming goes on to say the following statements:

Explaining why one guy was successful while the others weren't as much successful.

I have never heard the names of any of the other fellows at that table mentioned in science and scientific circles. They were unable to ask themselves, "What are the important problems in my field?" If you do not work on an important problem, it's unlikely you'll do important work. But the average scientist does routine safe work almost all the time and so he (or she) doesn't produce much. It's that simple. If you want to do great work, you clearly must work on important problems, and you should have an idea. Most great scientists know many important problems. They have something between 10 and 20 important problems for which they are looking for an attack. And when they see a new idea come up, one hears them say "Well that bears on this problem."

Now my question is: If I ask myself the question, What are the important problems in my field?, how should I expect to get an answer to this. Is it by deeply diving into literature and finding loopholes that exist. And even if I do find something worth working upon, how do I know it's an important problem of the field or a trivial subset of some other large problem?

I am sorry but I am unable to get myself a clear picture of what exactly is meant here. Will anyone please help me out with ideas.

Many thanks.

  • 1
    Well, yes, it is about delving deeply into a research area, and finding ways to push the boundary of knowledge. I would think of it as follows: if only I have X, then I can do Y, where Y is a very significant outcome; e.g., teleportation or infinite energy. X could be a widget/idea/concept/info. I would venture to say that X is what Hamming wants us to focus on. Shannon wanted X to get Y (error free transmissions), and he proved the properties (X) by which Y can happen. – Prof. Santa Claus Apr 2 '19 at 3:49
  • 2
    Hamming also said that "it's not the consequence that makes a problem important, it is that you have a reasonable attack". – Rodrigo de Azevedo Apr 2 '19 at 11:03
  • 4
    For some fields overview papers are published. These can bring you on the right track. Also a chat with a professor in the field should help to identify the important problems. – jos Apr 2 '19 at 11:21
  • 1
    I'd say that asking "how do I know it's an important problem of the field or a trivial subset of some other large problem" is a sign that you don't understand your field deeply yet. The question "What are the important problems in my field?" is a hard one, intended for senior researchers. The peers of Hamming could answer that question; invited talks sometimes propose answers to that; a principal investigator needs an answer to write a good grant proposal; but I think that most researchers publishing in some field don't actually have a solid answer to this, only a likely flawed opinion. – Peteris Apr 3 '19 at 22:28
  • @Peteris Yes, you are right. I am just starting out, beginning by reading articles in Deep Learning. My motive to read this seminar ( and others like it ) is to get an understanding from other people's experiences about the general process and practices in research. And here I came across this statement, so posted it out here to see what the community thinks it to be – Nimish Mishra Apr 4 '19 at 1:37
33

What are the important problems in my field?, how should I expect to get an answer to this. Is it by deeply diving into literature and finding loopholes that exist. And even if I do find something worth working upon, how do I know it's an important problem of the field or a trivial subset of some other large problem?

I would interpret this statement not so much as trying to look for "loopholes" in the literature, more as trying to identify what the core problems are that are holding back further advancement in your field, and what would need to change in order for them to become tractable. This requires you to deeply understand the existing work, so as to learn what exactly needs to change for your field to really jump forward.

However, I should warn you - Hamming’s approach is certainly a high-risk-high-reward one. If you implement this, you will almost certainly have to invest quite some work detailling and understanding a large number of difficult problems that you will in all likelihood make no progress on during your lifetime. Really important problems invariably don't have a straight-forward solution (or, in many cases, no solution at all), so even if you bring yourself into the best position to attack them in the right circumstances, there is little guarantee that these circumstances ever come to pass (or that you are the first to recognize if some recent development in the field makes one of "your" open problems tractable). That said, if you manage to address even one really big, important problem, your impact will indeed be higher than what you can achieve in decades of incremental research.

  • 3
    I suspect the OP simply wanted to say "holes" (as in "gaps in knowledge") rather than "loopholes". – Wolfgang Bangerth Apr 2 '19 at 14:11
  • @WolfgangBangerth Yes surely I meant gaps in knowledge. Am sorry for the confusing statement. – Nimish Mishra Apr 2 '19 at 17:25
  • Hey, worst case scenario, he might be able to get a meta-analysis paper out of his analysis of the problems his field is facing! ;) – nick012000 Apr 2 '19 at 23:38
23

Not everyone is a Richard Hamming or a John Tukey. They did what they did because they were brilliant. It's not like everyone can become them just by deciding to work on something different.

My advice is to keep a notebook and write down ideas. Occasionally do things like putting them in Excel and organizing them into categories (including multiple tags).

Personally, I WOULD find some safe work to do. That you know delivers results and that you can spend time on and get rewards from. Try to at least work up to, and around more fundamental ideas. But it is also good to "grind your pigments", not just jump to painting Mona Lisas.

  • 20
    +1 just for the first paragraph. The OP's quote is pure survivorship bias and nothing else. The big-time names got there almost entirely by being fortunate enough to be ridiculously intelligent (among already ridiculously intelligent people) and well-motivated. Everyone else got where they are by achieving to the greatest extent of their abilities. Not because they simply "did it wrong" or didn't pull themselves up by the bootstraps. That's wildly insulting. – zibadawa timmy Apr 2 '19 at 8:48
  • 2
    @zibadawatimmy This was exactly what I originally wanted to answer as well. Survivorship bias is an important concept to keep in mind whenever listening to talks of or getting advice by excessively successful people. – xLeitix Apr 2 '19 at 11:39
  • 14
    I wholly agree with the first sentence, but not the second. Hamming and Tukey were brilliant, but a significant factor in their sucess is the luck of being active when their fields were young. Brilliance is neither necessary nor sufficient for success. Someone else would have rediscovered Hamming codes and FFTs if they hadn’t. – JeffE Apr 2 '19 at 12:47
  • 3
    @JeffE I guess, a less fancy way to express your way to fame would be "try to get in on the ground floor on the next big thing" (of course nobody can really tell what the next big thing is going to be - and this is where survivor bias will strike again). – xLeitix Apr 2 '19 at 12:54
  • 2
    @xLeitix Or even more simply: “Work hard, and get lucky.” – JeffE Apr 2 '19 at 14:40
6

The important problems in each field are different, of course. Some of them are well known and well publicized. In CS, "Does P = NP" is one of those problems that every advanced student is introduced to. The same will be true of such problems in other fields. However, one of the reasons that such problems are important and well known is that they are very hard and that lots of people have tried and failed to solve them. There is normally a rich literature on such long-standing problems. But you aren't likely to solve them yourself. Some may even not be solvable at all.

Other problems are important, but unknown. They won't be known to be important until they are solved and the solutions are found to open up new lines of thought and inquiry. These are often solved purely by serendipity. Someone has an "interesting" thought and explores it. That leads to other interesting thoughts. If this work becomes known it may be post facto treated as important.

But a solo researcher is pretty unlikely to be the one to solve such problems, though it can happen.

Collaboration is the key

Important problems are most often solved by people who are not only able, and well trained, but also connected to other such people. In a research group, whether co-located or not, there are a tremendous number of ideas floating around and discussed. Some of these problems are big problems and some of them are just "interesting looking" areas of inquiry that haven't yet been mined for results. If you can get connected, somehow, to such a group then you will learn about those big, but not well-known, problems that might be worth attacking.

This is why research seminars at large departments are so valuable. People with a somewhat different viewpoint than your own can make suggestions, some of which can bear fruit. Doctoral dissertations often come from such seminars and some of them are important work on important problems, whether they provide a complete solution or not.

So, if there is advice to give here, my advice would be to look, first, for "interesting ideas" not important problems. Many of the important problems aren't known to be such so looking for them directly isn't fruitful. But find a group to work with and look at a lot of ideas. Some of them will, hopefully, turn out to be interesting. Hamming didn't work alone, of course.

And yes, what is "interesting" varies by field and with time.

3

While I think guest's answer is relevant, here is a counterpoint.

It is true that not everyone will be able to solve outstanding problems in one's field. However, (imo) a recipe for bad science is to specialize to the extreme and keep working on a narrow set of problems, interesting only to a tiny community that doesn't exchange ideas with the rest of the field.

Here's how to know if a problem is important : talk with other people in your field, see what interests them. Everyone has their own backgrounds and their own interests, but the more attention a problem draws, the more important it is.

The amount of literature is also relevant: if there are a lot of publications from well-established people all around the world on a certain topic, then it's interesting (at least to people from your field). If the only people publishing papers about it are 2 guys in an obscure university, who are only referenced by each other, and who wrote 15 papers on the same topic all published in the same journal, it's probably not a serious topic.

So in conclusion, for most people it may be unrealistic to want to only work on major problems. BUT it doesn't mean that you shouldn't try to tackle (reasonnably easier) problems that are of interest to your community, instead of focusing too much on "what you know you can do".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.