Questions related to research in which data and/or code is made publicly available so that others may verify the findings and build upon them.

learn more… | top users | synonyms

4
votes
2answers
924 views

When replicating results, what should I do when the author didn't use the method the paper said they did?

I am replicating a model to report on how well it does on a new evaluation metric I am proposing. The method/hyperparameters of the model are given in a paper about it, where it breaks all benchmarks ...
3
votes
2answers
127 views

How can we perpetuate reproducibility?

Two years ago, Roger Peng discussed in his blog the ramifications of requiring reproducibility for the first producers of research objects, but not for reusers. He then proposed What I think a ...
11
votes
5answers
20 views

Are virtual machines good for reproducibility and open science?

Virtual machines (VMs) provide a way for scientists to package not only scientific software, but also data, external dependences, and even entire operating system configurations, facilitating a ...
5
votes
4answers
223 views

Ethical considerations in publishing raw data

Apart from taking measures to maintain confidentiality, do I need my ethical approval to specify permission to publish raw data along with my study? I'm trying to facilitate reproducibility in my ...
12
votes
3answers
2k views

Open Source Software and Reproducibility in Computer Science

As a PhD researcher in computer science, my scientific results often take the form of algorithms, which in turn come with implementations. I am convinced that releasing these implementations under an ...
21
votes
5answers
3k views

What to do when the claims in a paper are proven to be wrong?

What can be done to correct an error that might be hiding a certain author's sole purpose of publishing a paper, no matter the soundness of their method? In a nutshell, the particular case that ...
13
votes
3answers
277 views

Where do you place the link to open data in a manuscript?

Assume you are writing a manuscript and you have uploaded your raw data to a relevant hosting site (e.g., The Dataverse Network, FigShare, etc.). Where in the manuscript should you indicate a link to ...
17
votes
4answers
385 views

Is there evidence that open source research software produces more uptake of methods?

In recent years there has been a drive to make science more open. This includes making the software used to perform research open source. The main argument in favour of this idea is that research ...
6
votes
1answer
117 views

How should I split my code and data between an open repository and the article's supplementary files?

I am working on a paper that involves some data and a piece of code to analyse the data. I want the results of the analysis to be reproducible and the code to be open. This concept of "reproducible ...
22
votes
5answers
2k views

Ignored Requests Open-Source Software from a Research Group

For more than 6 weeks now, I have been attempting to contact a post-doc or their (former?) PI to request access to either the source code or software of a tool that was published in BMC ...
5
votes
1answer
85 views

Are there any experiment reproducibility databases?

Quite often when I read a medical research article I wonder whether other researchers tried to reproduce the experiment. I could go through the list of papers that cite the article but I find this ...
7
votes
2answers
218 views

Pedagogical pros and cons of requiring undergraduate students to follow open/reproducible research practices?

I regularly teach an undergraduate course in which students are expected to complete projects that involve writing code. I also supervise undergraduates in other project and thesis work that involve ...
17
votes
3answers
1k views

Can I request the code behind a research paper from the author?

I am in the field of computer science. It is often the case when I am reading a paper I start to wonder, "Wow stunning results, however I would like to proof that" or "How did he exactly got these ...
9
votes
5answers
207 views

Are there any proposals to make sharing code and data with publications a requirement?

Depending on the project, many researchers have to rewrite code and reproduce datasets from previous works in order to perform experimental evaluations. Although some dataset are understood to contain ...
18
votes
3answers
1k views

What should I do if I cannot reproduce experimental published results

I found a useful advice if I cannot reproduce computational published results in (What should you do if you cannot reproduce published results?). However I would like to ask the same situation with ...
3
votes
1answer
324 views

Forum for discussing research paper

I am trying to implement an algorithm given in a paper. I have some questions which I would like to clear out since implementation requires complete clarity on all the concepts/algorithms/arguments ...
15
votes
6answers
840 views

Licence that converts to GPL after publication but is copyrighted prior to publication

I would like to post an repository on github that includes data and analysis code in R. The analysis and data forms part of a journal article submission. I'm happy for people to view the code and data ...
14
votes
2answers
634 views

As a reviewer, how much raw data or code should you request?

The topic of reproducible research is attracting quite some press these days, yet much remains to be done. In this spirit, I am asking what can be a reviewer's role in this — I'll explain fully below. ...
9
votes
2answers
829 views

How does a PhD student go about doing a meta analysis of a research question?

I just thought that I would put this question out here because I am currently working on a meta analysis of a particular research question in my field (HCI/information science). Moreover, this ...
16
votes
3answers
232 views

How to replicate others' results quickly?

Lately, I've been getting the impression that before I can conduct cutting edge research, I have to first master state of the art methods/techniques. Often, this requires spending (i.e. wasting) lots ...
13
votes
5answers
563 views

Has it ever happened that one paper's findings were contradicted by another?

Since reviewers don't check the experimental results by trying to reproduce the experiment, is it possible for someone to submit a paper which basically says "Method X was proposed in paper Y and ...
51
votes
8answers
2k views

What should you do if you cannot reproduce published results?

I am trying to reproduce published results in a paper. Those results come from numerical simulations. The original authors and I do not use the same software, and theirs is proprietary (I don't have ...
25
votes
6answers
727 views

Reproducible Research :: Proprietary or Open Source Software?

For a long time, I had been using open source software for my work to boost "reproducible research". I believed that if I made my codes open source and the softwares to run those codes in were open ...
29
votes
3answers
2k views

Requesting raw data from previously published research

I am working on a mathematical model to help explain some experimental results and generate new hypotheses. Unfortunately, I have neither the resources nor interest to gather experimental data, but in ...
9
votes
1answer
245 views

How long should I register a domain name that will be referenced in scientific publications?

I have a website (with data and modeling interface) that will be referenced in a series of scientific publications that are beginning to be published. Currently the website is along the lines of ...
28
votes
5answers
714 views

Data publication basics - where, why, how, and when should I publish my unpublished data?

Many researchers have unpublished data. Some of this data may never be published as a manuscript. But I would like to make scholarly contributions of data that I have no intent on publishing, e.g. by ...
23
votes
3answers
489 views

How to get the data to reproduce a published result?

I'm studying published journal articles on my topic, and I'd like to reproduce their analysis. How can I get a copy of their dataset and enough specifications to reproduce their numerical results? ...