How to replicate others' results quickly?

Question

Lately, I've been getting the impression that before I can conduct cutting edge research, I have to first master state of the art methods/techniques. Often, this requires spending (i.e. wasting) lots of time trying to replicate the results that others have already established (i.e. make sure that I can do what others have done first before moving on to other territory). When I try to master a new technique, I often find that it is extremely difficult because

1. Research papers don't seem to disclose enough details to replicate their results easily.
2. There are no advisors/colleagues that are familiar with these methods
3. Emailing the corresponding authors with questions about specific details about their research tend to go unanswered.

The only advice i've been given thus far is simply to read and re-read the research paper and dissect every little detail until I figure it out. But are there any other general strategies I can use to replicate others' results more quickly? In particular, I'm looking for strategies for replicating numerical results that require very careful implementations of mathematical algorithms.

Answer 1

As JeffE points out, trying to reproduce existing results before embarking on new endeavors is not a waste of time. It can be particularly useful, especially in tracking down issues that you have within your own code—or, perhaps, in the existing literature. My most-cited paper, and much of the research work currently ongoing within my group, came about precisely because I could not replicate an existing paper. As it turned out, there were some significant methodological problems in the existing literature, and this led to a fair bit of digging around, plus a lot of careful analysis to demonstrate what had happened and why. As I've also said, the issues we uncovered there indirectly led to a different line of inquiry that has already led to half a dozen additional manuscripts.

If you are determined to speed up the process, however, you might try to consider simplified versions of the research cases presented that can validate what you're working on. For instance, can you find a "prototype" problem whose solution would be known with your method as well as the existing methods? Can you find one or two test cases which can test most of what you're looking for? Can you devise a new problem that compares the two better than the existing cases, and is also simpler to execute?

Answer 2

The fact that you can't quickly and effortlessly do this is exactly why this is not wasting lots of time, assuming that you do in fact mean to build upon this very same research. You're learning how to solve problems in this arena, given an outline that you know works, but insufficient detail to actually implement it fully.

Other than that, for numerical results you simply need to do lots of testing to make sure at each stage the calculation comes out to be what you expect (and tests any important corner cases). For example, if something's piecewise polynomial, and you drop three points at random into every piece, and it's accurate to within machine precision, you've almost surely got it right, and if it's not you've almost surely got it wrong.

Now, once you're already well-versed in a field, you may write a paper which referees want you to compare to some other new result in the field. Replicating someone else's poorly-described new methods really can be a waste of time; you don't learn anything except that they left out a lot of fiddly details. (Editors may be sympathetic and let you skip it or e.g. use a common dataset instead.)

Answer 3

I must agree with the answers posted above - it most certainly not a waste of time at all - my current supervisor explained to me that in a way, it is analogous to the saying "Learning to walk, before you can run" - if you can master what has been researched before, particularly learning the methodologies used - then you would have a greater chance of improving these methods and potentially further refining the results.

Also, consider that when the papers don't give much away, that this is a prime opportunity to develop your problem solving skills within the field that you are researching. I am doing the same, with a unique Android app as part of my research, I am entirely self taught from learning how other apps work.

Don't give up - keep on persevering, it is worth it!