Take the 2-minute tour ×
Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

The dominant narrative on how research "should be" done is to begin with a literature survey and then make your own contribution.

But I get the impression that it often happens the other way around - at least in fields such as CS where the cost of knocking some lines of code together is low (compared to, say, a 3 year medical testing programme). Creativity knows no rules.

  • The obvious risk of this approach is that the research is subsequently found to be old hat, or better versions of the same thing exist already, or it commits an error already widely known in the field.

  • The reward, on the other hand, is that such creation might never have happened if the requirement of surveying the literature first was always met. (Indeed some grad students have been known to quit before finishing the lit review their supervisor asked for - maybe they'd have done better if allowed to exercise their creative nature first?).

So I would conclude that it's an acceptable strategy when the individual is happy with that risk/reward balance (provided they already know enough about what they're doing to have some inspiration in the first place, that is).

Are there other arguments/perspectives I've missed here?

UPDATE

There are a lot of very good answers here and there is no one right answer. I'm not quite sure how to handle that. In my experience this is the kind of thing that tends to lead to questions being closed as "not constructive" on SE sites. I'm glad nobody has done so yet because I think the full range of opinions expressed below is a very worthwhile thing to read through. Thanks all for your contributions.

share|improve this question
28  
Frank Westheimer, a chemistry professor at Harvard, once famously opined: "A couple of months in the laboratory can frequently save a couple of hours in the library." –  aeismail Apr 4 at 13:41
2  
surveying the literature is research... –  vzn Apr 4 at 13:42
16  
A side remark, from Paul Graham's Hackers and Painters essay: "Scientists don't learn science by doing it, but by doing labs and problem sets. Scientists start out doing work that's perfect, in the sense that they're just trying to reproduce work someone else has already done for them. Eventually, they get to the point where they can do original work. Whereas hackers, from the start, are doing original work; it's just very bad. So hackers start original, and get good, and scientists start good, and get original.". –  Piotr Migdal Apr 4 at 13:57
5  
That said, even in science, I prefer the hacker's approach. (I learn more by trying and making mistakes than reading a polished version of results.) –  Piotr Migdal Apr 4 at 14:08
5  
suresh, you might actually state what you think is wrong about his assertions, rather than merely vacuously asserting they are wrong. –  vzn Apr 5 at 2:39
show 5 more comments

9 Answers 9

up vote 10 down vote accepted
  • One point I don't see so far is that a quick experiment (in lab or in silico) can result in showing that the idea wasn't that great after all. And in that case, it is rather unlikely that literature says anything about it at all.

  • Another point (rather in line with @Andy W and @badroit's answers) is that spending some effort of your own on the problem may be needed in order to find the relevant literature.
    (I've had problems that I eventually solved myself, and where only the solution I obtained allowed me afterwards to find some relevant literature in a different field - simply because only the solution yielded successful search keywords)

  • Of course, students solving old and known problems is often called "education" and considered necessary, not a waste of time...

  • Last but not least, a few hours in the library often still make months in the lab necessary...

share|improve this answer
    
I think your first point is very good indeed - exploring ideas for yourself helps you avoid being a victim of publication bias. (If they're cheap to explore, at least). –  Sideshow Bob Apr 7 at 15:29
add comment

I don't entirely agree with the answer "No". This may be because my field is mathematics, though. Sure, not looking at literature beforehand means you are at risk of re-doing something that was already known and it's never nice to discover after spending lots of time working on something that it was already out there.

On the other hand, when you come across what seems to be a good question, I don't think it's such a bad idea to start by asking yourself "Can I answer this myself?" and not rush immediately to see what other people have already done in relation to this question. I see a possibility of positive outcome to not being aware of previous results for a significant time (of course not to the point of being on the verge of publishing and discovering the day before submission a very similar paper...).

As ff524 pointed out in a comment, "if you are a researcher, [...] you already have some idea what's been done before". If you find an interesting question and don't know about an answer, it may be because it does not exist, but it may also happen that it does and has been overlooked before. Maybe it came out at a time when it didn't meet the interests of the community, or it came from another community, or the communication on the result was not so good, or it didn't look interesting because the strategy was inappropriate. If you find this result early on, you will likely drop the question, not advertise it and the result (although of interest to the community) will remain unknown. If you start searching the answer without knowing about the previous result, it's likely that you will not follow the exact same path and that can be good.

  • Maybe your way is more "natural" and makes the result fit nicely in a developing theory that didn't include it before. It's not such a bad idea to provide a new way of looking at something, even if the result per se is not new.
  • Maybe your way is more fruitful, in the sense that during your quest for the answer to the original question you will come up with new interesting questions which did not stem clearly from the way the result was previously established.

It's still risky, but in research you constantly invest your time in stuff you don't know the outcome of.

share|improve this answer
3  
Thank you for leaving this answer. I felt very similarly, and I am glad to see an eloquent expression of it. –  Pete L. Clark Apr 4 at 19:33
add comment

Research before surveying the literature - is it ever a good idea?

If I reword that as:

Independent problem-solving before surveying the literature - is it ever a good idea?

Yes. I think in most cases it is a good idea to make an "initial effort" (the definition of "initial effort" depends on the scenario) to independently solve a problem before checking the literature. If you have not sat down and thought about the problem yourself, then:

  1. you may not readily understand the solutions (if any) proposed in the literature unless you've worked through them yourself

  2. you may not know the extent to which solutions are appropriate if you have not tried to apply your own ideas or thought of alternatives

  3. you may not know what to search for ... the core problem may not be as it initially seems but may reduce to something more well-known

  4. your own ideas on the problem may be influenced into the narrower-scope of what has already done (it might stifle creative problem-solving in the broader sense)

Of course, it is important to avoid working on a problem for months without referring to the literature at any point.

But certainly for a student, I would encourage them to think about and frame solutions for a problem strictly before they do a literature survey! Once they show understanding of the problem and of some approaches then I think they're ready to read papers ... doing it in this order, they're much better equipped to recognise relevant papers and interesting approaches when they see them.

share|improve this answer
add comment

To a certain extent this creates a false dichotomy. My mind is always turning with new ideas, and I may jot them down on paper and give then a bit of thought before I go and see if it is old hat with prior literature. The bit of thought may be a few hours of deliberation, or some simple experimentation or writing a few pages of a paper.

You need a general knowledge of the field to be able to understand how to make a contribution. You then have an idea, read on that specific prior literature, and this will often refine your idea. The more specific your idea becomes the easier it is to find prior literature and see if/how it is different. It is not a one before the other - they are constant and reciprocating.

The consumption of the pertinent scientific literature though is forever and endless.

share|improve this answer
1  
I very much agree with this iterative view on science. –  cbeleites Apr 5 at 19:19
add comment

While doing my Master's thesis my advisor said that (up to the wording):

It is better to spend the same time solving a problem than searching for its solution.

You learn more (compare: solving a problem in class vs googling for its solution) and more fun.

However, the question is:

  • What is the time overhead (i.e. how much longer does it take to solve it)?
  • Do you get the big picture?

If solving takes too much time, you may end up learning a lot but not doing new research. Also, without feedback (literature review or even better asking experts if the problem is novel and worth pursuing) you may end up with:

  • solving already solved problems (perhaps with much more general methods)
  • or solving problems no one cares about.

In any case, any sensible research strategy involves both.

share|improve this answer
2  
+1 also for this answer. Your advisor put it remarkably well. Since there is a strict inequality, by continuity it must be the case that there is some positive number epsilon so that it is better to take 1+epsilon as much time solving the problem yourself than searching the literature for its solution. What you have to decide as an academic is what is your cutoff value of epsilon. I would be quite happy to take epsilon = 1, for instance; but epsilon = 9 would be very frustrating to me. –  Pete L. Clark Apr 5 at 22:15
    
@PeteL.Clark For technical calculations epsilon may be negative (from my experience). In other cases it is usually positive, but not always (as it is not only about reading, but digging for result in papers; sometimes in different notation, sometimes - among many similar yet not equivalent problems, etc). –  Piotr Migdal Apr 8 at 15:29
    
The statement for negative epsilon is implied by your advisor's statement. But anyway, I have many times had the experience that it was quicker to reinvent something than look for it in the literature. However, this partly depends on one's skill in searching the literature, which in my experience is something that mathematical scientists are rarely taught and are therefore much less good at than most other kinds of academics. –  Pete L. Clark Apr 8 at 19:41
add comment

A few thoughts. This is clearly a delicate balancing act by the individual and also an area of convention passed down from teacher to student in different fields. One of the main roles of an advisor is to keep the student on track and make sure they are not going off in "unproductive" directions. "Unproductive" involves extremes. The young researcher may be spending a lot of time experimenting without looking at the literature, or too much time looking at literature and not experimenting.

A young researcher who spends a lot of time on an idea and then the advisor quickly shoots it down as previously explored may find his/her balance is slightly off and need to "recalibrate". But also realize this is a common scenario that has happened to all fledgling scientists, and even senior ones. Research and experimentation are a feedback loop which of course go "hand in hand". One must be continually/flexibly "jumping between both worlds" at all stages of development.

Yes, each scientific field probably has slightly different conventions. Some fields are more empirical than others. Some are "younger", and yes there is some case to be made that computer science is a younger field than others, which means that the balance may be different in this field. Also, there is some case to be made that empirical research in CS is underexplored at this moment in the larger scheme.

However, computer science is a somewhat unique field in that, yes, the individual may be able to do scientific experiments without expensive laboratory equipment independent of his research laboratory, on "his/her" own time, and gain significant understanding/insight toward their research thesis.

share|improve this answer
    
see also recent breakthroughs in CS empirical research & many refs –  vzn Apr 4 at 13:58
    
Advertising and linking to your own blog without disclosure, particularly as in this case when it doesn't address the question directly, is spamming. Please don't do that. –  EnergyNumbers Apr 4 at 19:57
3  
it is manifestly not "advertising"; there is no product; nothing is being "sold"; the questioner asked about empirical research in CS specifically & the page has many recent refs/research/leads on the subj (did you even look at it?) the defn of "spamming" varies widely but there is no officially accepted stackexchange criteria, & preferrably neither should there be, and users routinely use the word merely to complain/criticise virtually any offsite links whatsoever.... my affiliation with the site/se accts is seen in the url itself & openly/clearly stated on the "about" page on the site. –  vzn Apr 5 at 2:23
    
ps forgot to mention there are numerous related se cross-site Q/A links surveyed on that page also (& thx others for support on this). however must concede that alas it is not uncommon on se for some to refer to mere related blog links/essays/writings as "advertising/spam/selling" even by mods on misc sites =( –  vzn Apr 6 at 16:50
1  
@EnergyNumbers: it's a good on-topic link. Perfectly acceptable. –  smci Apr 7 at 1:59
add comment

I agree with your conclusion that it is an acceptable strategy when the individual is happy with the risk/reward balance. My supervisor used to always encourage his students not to do a literature survey but instead work on a concrete problem. Some students liked this approach, others did not like it and left to somewhere else within few months and still did very well in their research. I personally was undecided.

An obvious reward of this approach is that you may be able to publish something interesting within a year from the start of your PhD study. On the other hand you may end up with a negative result that can not be published and thus you may decide to change topic.

Spending a year or so on a literature survey may seem like a waste of time, however it can really help in the process of building an overall understanding of the field and boost your confidence and ability to discuss with more experienced people.

share|improve this answer
2  
not to do a literature survey but instead work on a concrete problem - why is it either/or? –  ff524 Apr 4 at 13:39
1  
As I mentioned, I was undecided, so basically did both simultaneously. This is easier to do when you are in your second or third year as you are already familiar with the basics. What I found is that working on problems require a degree of dedication and is time consuming. On the other hand, this maybe just my inability to multi task! –  2cool4school Apr 4 at 14:06
add comment

Knowing the state of the art is necessary, as well as having a clear definition of the problem that has to be solved.

Otherwise the student can:

  • make old know mistakes, as you pointed
  • solve old problems that have already been solved
  • worse than that, solve problems nobody cares about and where the solution is completely useless (specially in middleware that will not be used by any top layer or lower layer).

In short, it can be a complete waste of an arbitrarily long time.

Whether this is done by the supervisor pointing at a few selected papers or the students reading in a uninformed way trying to understand what they got into is a different matter.

IMHO, the supervisor should guide the student in the first years. A few minutes to make a reading list can save months of reading not-so-interesting things, being lost in the broadness and vagueness of a not properly defined state of the art/problem/area, and a lot of frustration and misunderstanding.

share|improve this answer
add comment

I would argue that it only becomes "research" when reading up on existing literature is involved. Basically, if inspiration hits you and you just fire up your IDE to test it, you are not aware if what you are doing is in any way novel. That's not really the nature of research.

Edit: (I should add that even if something is not novel in a scientific sense, it can still be extremely interesting and cool to do - however, it will likely never lead to a paper)

That is not to say that a researcher is never allowed to just follow her/his inspiration. It has certainly happened to me more than a few times that I had a crazy idea and just ran with it. Oftentimes, the quickly led nowhere and I just forgot about it. Sometimes, initial results proved interesting and then I went ahead and did proper research before going further.

Edit 2: I have just noticed that I did not actually answer your question:

Research before surveying the literature - is it ever a good idea?

No, not really. If you are doing "research" (in the sense that you are hoping to publish it, and not doing it out of personal interest, or as a training exercise, etc.), the only senseful way is to start with a literature review. One needs to keep in mind that in most fields, most of the "low-hanging fruits" (i.e., obvious ideas) are long solved. So just going ahead and doing something in the hopes that it may be novel has a significant error rate.

share|improve this answer
6  
Actually, if you are a researcher you are always reading the literature, attending conferences, and talking to other people about work related to your general research interests. When inspiration hits, you already have some idea what's been done before (unless it's in a new area for you). The formal literature survey just clarifies the exact bounds of what's new. –  ff524 Apr 4 at 13:24
    
Yes, with the important words being "some idea". I certainly pride myself on having a good overview over the state of the art, and still happens plenty that I suggest a direction that my students find out has already been done / tried before. –  xLeitix Apr 4 at 16:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.