40

I'm studying published journal articles on my topic, and I'd like to reproduce their analysis. How can I get a copy of their dataset and enough specifications to reproduce their numerical results?

In my specific case, I'm studying this work: Stecklov et al. (2005), published in Demography, and they're using PROGRESA datasets which are publicly available, but when I try to reproduce their results, all the base statistics are off by a bit, and the regressions come out a bit different. Probably I'm just including or excluding a few different cases, but what can I do to identify the difference?

Improve this question
3
  • 16
    I'm not sure how we can help. The authors of the paper are in a better position to help, and most authors will respond to a polite request. It's likely that there were some undocumented steps in data handling.
    – Jason B
    Commented Oct 24, 2011 at 12:59
  • 1
    I was trying to replicate a paper of my professor by implementing the algorithm he mentioned in his paper. I proved that his algorithm written on the paper was wrong, and he accepted it too as a typo. Sometimes people love waiving their hands. I don't think any reviewer sits down and replicates the paper.
    – user2792
    Commented Oct 6, 2012 at 14:58
  • You could also take a look at this related question which has several useful answers about requesting data from published research.
    – Jez
    Commented Dec 11, 2012 at 12:16

4 Answers 4

Reset to default
21

Email the author nominated for communications, and ask them for a copy of their dataset and enough specifications to reproduce their numeric results. As a backstop, ask for the specific criteria they used to exclude cases. Consider offering co-authorship on any papers that will result from your analysis, as incentive. Give them enough context about you & your own work, so that they know your enquiry is serious, and that you're looking to constructively build on their work, (and not to stitch them up!)

Improve this answer
1
  • 20
    Co-authorship seems a bit extreme, and possibly unethical.
    – Ari B. Friedman
    Commented May 4, 2012 at 11:34
13

Read carefully, underline everything it says about anything to do with the data, but it is likely you won't get it quite right. For Labor we were assigned to reproduce "Tax Incentives and the Decision to Purchase Health Insurance: Evidence from the Self-Employed" by Jonathan Gruber and James Poterba (QJE, 1994) and "Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size" by Joshua D. Angrist and William N. Evans (AER, 1998). Try as we might, no one in the class ever got exactly the same things as in the papers. It was always about right, and enough people did it independently that we trusted we were doing the right thing. After a while the small differences stopped bothering my classmates and I.

Improve this answer
2
  • Out of curiosity, what class was that where you replicated papers?
    – dchandler
    Commented Oct 24, 2011 at 4:53
  • 1
    The first class of a PhD labor sequence.
    – Bitsy
    Commented Oct 24, 2011 at 14:50
11

Replication is an incredibly important part of academia. Don't forget that it's possible you are failing to replicate because the original authors did something wrong! You should absolutely email them, but keep in mind that failure to replicate can be publishable, provided your differences are significant and you can couch couch our results in current theory.

Have a look at Gary King's "Replication, Replication"

Improve this answer
-1

My first thought is that you may ask help in support team where this dataset is placed. Also, you may ask and find out what is wrong with the author. Some data may not be available or partially shown, although access is open. Perhaps this was done for reasons of plagiarism.

Improve this answer

You must log in to answer this question.