How to find research gap? [closed]

Question

I would like to know if there is an easy way to find the research gaps and the new possible development track in the subject, apart from reading hundreds of articles. Indeed I would like to elaborate a research proposal without some additional added value (not doing what others have already done) Thank you!

Answer 1

I'd like to chime in with an important caveat to what you said - it appears you want to avoid reading "hundreds of articles". The purpose of reading an article is not just to understand what is left to be done. Rather, it is important to understand the field, its lingo, what has been solved, common research methods, expected level of discourse and analysis etc. Although it has been said that academic papers are how scientists obscure their research from one another, it is also true that academic papers are how new researchers get introduced to the field and existing researchers keep abreast of it.

When I was a masters and PhD student, I would spent a lot of time reading the introductions of papers, since it was there that I found the most interesting overview of the field. As I progressed, I skipped those sections because, for the most part, they became all the same ("X is a problem, however, Y. Therefore, Z is needed").

In addition, papers often have a background section and their own reference lists, many of the references are absolutely crucial to understanding the field. I've read many bad papers that turned me on to a set of gold references.

Research problems are not found simply by looking for things no one has done before. That is just chasing novelty and the problem often is that novelty isn't enough. For instance, it's true that no one has tried to put a motor into a pork chop, but that doesn't make it a relevant robotics research topic. Reading a paper to find research ideas should be more like "Oh, they did XYZ. Interesting, because that method has a deficiency when dealing with the following class of outliers. I wonder why they didn't explore that?"

Answer 2

That depends a bit on the field, especially on its "maturity" and on the number of people interested in it, not just currently, but in the past as well. In a new, fresh, hot field it will likely be easier than in an old, well travelled one since people may be skipping a lot as they push on.

But for most people, no, there isn't an easy path. You gotta do the grunt work to root out the gems.

You can ease the task, however, if you have a group of collaborators who meet fairly regularly to kick ideas around. Some of them will be trivial, others very hard, some just right (three bears problem). For a student, the advisor, who should have more breadth of knowledge in the area is a useful resource.

Experience also helps, but it takes hard work to get that experience. If you have insight into a range of problems the missing chinks might be more visible.

And it takes more that "reading". Careful, critical, reading is needed.

Answer 3

Focus on your own curiosity and knowledge gaps --- some of these will be research gaps

In my experience, one of the most effective ways to find and plug research gaps is to ask yourself questions that are of interest to you as you read through the material in a field and play with problems for your own enjoyment and interest. When you read a paper, it ought to spark a few questions you have that are of interest to you, and you will probably be motivated to play around with the problem and see if you can figure out the answer. This will also lead you to see if these questions are already answered in the literature, and if so, whether the answers in the literature are the same as your own answers. If your question is already answered in another paper, you then read that other paper and it then sparks new questions, etc. If your questions are not answered (or if they are answered in a different way, such that your own alternative answer is a useful new perspective) then you have found a gap in the literature relating to a question that is of interest to you, which then gives you an opportunity for a publishable paper.

In conducting this exercise, I find it best to ignore consideration of wanting to find research gaps and publish things, and instead just focus on satisfying my own curiosity about problems. Plenty of times this leads me to solve problems that I initially think are original work, but which turn out to be re-discoveries of material that is already published (for some examples, see this related answer). That is okay, because it was still interesting to work on the problems and it is heartening to rediscover valuable results in your field.

You should not impose preliminary limits on how much you ought to have to read to perform this process. If you have an aversion to reading a large number of papers in your field then it suggest that perhaps this field is not something you're passionate about, so a research career in that field is probably a poor choice for you. In my own field (probability/statistics/data science) I can happily read hundreds of papers because I find them interesting and they lead me to interesting problems that I can toy around with for my own amusement. (In practice I would probably start toying with problems before reading even ten papers, let alone one-hundred. Comprehensive reading of the literature tends to occur when I'm doing a literature review after having found a specific problem I want to write about.)

---

An example: At the risk of being self-ingulgent, I'll give you an example of a recent paper I've written that came out of this kind of inquisitive process. (Some field-specific jargon used here, but hopefully you get the idea.) A few years ago I got interested in occupancy distributions in probability, which came out of asking a simple question about random sampling. This led me to an interest in a probability distribution called the "classical occupancy distribution", so I started reading a bunch of papers about this distribution. One thing I noticed in the papers is that they all gave the mean and variance formula for the distribution, but did not give formulas for other properties of the distribution called the skewness and kurtosis. So already I had an immediate question of interest to me --- what are the formulae for the skewness and kurtosis. I also noticed that the papers mentioned that the distribution converges to a normal distribution under broad conditions (so you can use the normal distribution as an approximation in some cases), but the papers did not really attempt to describe how fast this convergence occurs and how accurate/inaccurate this approximation would be. So I then had a second question of interest to me --- how accurate is the normal approximation as I change the distribution parameters, and how large do those parameters have to be before we get really good accuracy? Finally, I had some questions about computation. The distribuion can be computed recursively, but this is slow when the parameters are large, so it occurred to me that you could compute recursively when the parameters are "small" but then switch to the normal approximation when the parameters are "large". But where is a good dividing line for this change in computation method --- i.e., what is a good point where you should change from recursive computation to an approximation method?

As I read through more papers on the topic it became evident to me that these questions did not have answers in the existing literature, so I worked on them myself, mostly out of personal interest and a desire to compute the distribution effectively. Had I worked on this and figured out the answers, and then found out they weren't novel, that would have been okay, because playing around with those problems is interesting and good fun. But in this particular case, once I'd figured out the answers I then had the material for a publishable paper that plugs a research gap (O'Neill 2020). This paper answered the above three questions I had asked myself while I read other papers in the field. I've since done more playing around with extensions and variations to this occupancy distribution and I've found more interesting questions that were not answered in the existing papers, so this has led to more publications in the field. This is the type of process that I find leads to my research --- reading papers, asking myself questions, and toying around with problems.