Questions related to research in which data and/or code is made publicly available so that others may verify the findings and build upon them.

learn more… | top users | synonyms

21
votes
4answers
2k views

Why are papers accepted even if they don't release code or data to allow reproducibility?

Recently, we had a lecture about Reproducible research on of the slides was: Reproducibility: start from the same samples/data, use the same methods, get the same results. Replicability: ...
1
vote
1answer
49 views

How to identify studies that should be replicated?

In psychology voting on which studies should be replicated is established on a website. The ReplicationWiki (that I founded) offers a voting option for studies in economics and related fields, but it ...
12
votes
2answers
463 views

How do you validate a scientific paper that uses a non-public cell/animal line?

Recently my lab published a paper that uses a mouse line from another lab. Without going into too much detail since not everyone here is a Biologist, this other lab has a mouse with a reporter gene ...
18
votes
5answers
4k views

Can I trust a paper that hides implementation details?

In Computer Science, sometimes we're assigned to reproduce the results presented by a paper that already have been published. In many cases (most of them) the code is not available, making the task ...
14
votes
2answers
273 views

I've reproduced a paper: where shall I share the code and findings?

The majority of papers don't share their code. However, I sometimes want to reproduce the results, and therefore re-code the experiments to try to reproduce the papers. Where shall I share the code, ...
4
votes
2answers
951 views

When replicating results, what should I do when the author didn't use the method the paper said they did?

I am replicating a model to report on how well it does on a new evaluation metric I am proposing. The method/hyperparameters of the model are given in a paper about it, where it breaks all benchmarks ...
3
votes
2answers
142 views

How can we perpetuate reproducibility?

Two years ago, Roger Peng discussed in his blog the ramifications of requiring reproducibility for the first producers of research objects, but not for reusers. He then proposed What I think a ...
11
votes
5answers
46 views

Are virtual machines good for reproducibility and open science?

Virtual machines (VMs) provide a way for scientists to package not only scientific software, but also data, external dependences, and even entire operating system configurations, facilitating a ...
5
votes
4answers
273 views

Ethical considerations in publishing raw data

Apart from taking measures to maintain confidentiality, do I need my ethical approval to specify permission to publish raw data along with my study? I'm trying to facilitate reproducibility in my ...
13
votes
3answers
2k views

Open Source Software and Reproducibility in Computer Science

As a PhD researcher in computer science, my scientific results often take the form of algorithms, which in turn come with implementations. I am convinced that releasing these implementations under an ...
21
votes
5answers
3k views

What to do when the claims in a paper are proven to be wrong?

What can be done to correct an error that might be hiding a certain author's sole purpose of publishing a paper, no matter the soundness of their method? In a nutshell, the particular case that ...
13
votes
3answers
294 views

Where do you place the link to open data in a manuscript?

Assume you are writing a manuscript and you have uploaded your raw data to a relevant hosting site (e.g., The Dataverse Network, FigShare, etc.). Where in the manuscript should you indicate a link to ...
18
votes
4answers
473 views

Is there evidence that open source research software produces more uptake of methods?

In recent years there has been a drive to make science more open. This includes making the software used to perform research open source. The main argument in favour of this idea is that research ...
7
votes
1answer
130 views

How should I split my code and data between an open repository and the article's supplementary files?

I am working on a paper that involves some data and a piece of code to analyse the data. I want the results of the analysis to be reproducible and the code to be open. This concept of "reproducible ...
22
votes
5answers
2k views

Ignored Requests Open-Source Software from a Research Group

For more than 6 weeks now, I have been attempting to contact a post-doc or their (former?) PI to request access to either the source code or software of a tool that was published in BMC Bioinformatics....
6
votes
3answers
119 views

Are there any experiment reproducibility databases?

Quite often when I read a medical research article I wonder whether other researchers tried to reproduce the experiment. I could go through the list of papers that cite the article but I find this ...
7
votes
2answers
280 views

Pedagogical pros and cons of requiring undergraduate students to follow open/reproducible research practices?

I regularly teach an undergraduate course in which students are expected to complete projects that involve writing code. I also supervise undergraduates in other project and thesis work that involve ...
25
votes
4answers
3k views

Can I request the code behind a research paper from the author?

I am in the field of computer science. It is often the case when I am reading a paper I start to wonder, "Wow stunning results, however I would like to proof that" or "How did he exactly got these ...
9
votes
5answers
234 views

Are there any proposals to make sharing code and data with publications a requirement?

Depending on the project, many researchers have to rewrite code and reproduce datasets from previous works in order to perform experimental evaluations. Although some dataset are understood to contain ...
18
votes
3answers
1k views

What should I do if I cannot reproduce experimental published results

I found a useful advice if I cannot reproduce computational published results in (What should you do if you cannot reproduce published results?). However I would like to ask the same situation with ...
4
votes
1answer
548 views

Forum for discussing research paper

I am trying to implement an algorithm given in a paper. I have some questions which I would like to clear out since implementation requires complete clarity on all the concepts/algorithms/arguments ...
15
votes
6answers
860 views

Licence that converts to GPL after publication but is copyrighted prior to publication

I would like to post an repository on github that includes data and analysis code in R. The analysis and data forms part of a journal article submission. I'm happy for people to view the code and data ...
15
votes
2answers
773 views

As a reviewer, how much raw data or code should you request?

The topic of reproducible research is attracting quite some press these days, yet much remains to be done. In this spirit, I am asking what can be a reviewer's role in this — I'll explain fully below. ...
10
votes
4answers
1k views

How does a PhD student go about doing a meta analysis of a research question?

I just thought that I would put this question out here because I am currently working on a meta analysis of a particular research question in my field (HCI/information science). Moreover, this ...
18
votes
3answers
245 views

How to replicate others' results quickly?

Lately, I've been getting the impression that before I can conduct cutting edge research, I have to first master state of the art methods/techniques. Often, this requires spending (i.e. wasting) lots ...
13
votes
5answers
585 views

Has it ever happened that one paper's findings were contradicted by another?

Since reviewers don't check the experimental results by trying to reproduce the experiment, is it possible for someone to submit a paper which basically says "Method X was proposed in paper Y and ...
54
votes
8answers
2k views

What should you do if you cannot reproduce published results?

I am trying to reproduce published results in a paper. Those results come from numerical simulations. The original authors and I do not use the same software, and theirs is proprietary (I don't have ...
26
votes
6answers
780 views

Reproducible Research :: Proprietary or Open Source Software?

For a long time, I had been using open source software for my work to boost "reproducible research". I believed that if I made my codes open source and the softwares to run those codes in were open ...
31
votes
3answers
3k views

Requesting raw data from previously published research

I am working on a mathematical model to help explain some experimental results and generate new hypotheses. Unfortunately, I have neither the resources nor interest to gather experimental data, but in ...
9
votes
1answer
285 views

How long should I register a domain name that will be referenced in scientific publications?

I have a website (with data and modeling interface) that will be referenced in a series of scientific publications that are beginning to be published. Currently the website is along the lines of ...
29
votes
5answers
779 views

Data publication basics - where, why, how, and when should I publish my unpublished data?

Many researchers have unpublished data. Some of this data may never be published as a manuscript. But I would like to make scholarly contributions of data that I have no intent on publishing, e.g. by ...
29
votes
3answers
837 views

How to get the data to reproduce a published result?

I'm studying published journal articles on my topic, and I'd like to reproduce their analysis. How can I get a copy of their dataset and enough specifications to reproduce their numerical results? ...