Questions tagged [reproducible-research]
Questions related to research in which data and/or code is made publicly available so that others may verify the findings and build upon them.
70
questions
1
vote
0answers
45 views
How do I find the ML model referenced in a specific paper when the authors don't provide it?
I'm trying to locate the architecture of an LSTM neural network as described here, but the authors don't provide a repository for other academics.
From arXiv, I see more than a dozen other researchers ...
2
votes
1answer
75 views
Reproducing experimental data from published work
I am doing theoretical study of a material and in my recent work I have compared my calculated results with experimental results from another published work. During review, the reviewer has mentioned
...
3
votes
1answer
117 views
Google Scholar Syntax quirks: results depend on order of search strings [closed]
I am working on a systematic literature review and noticed something quite disturbing since I want to base my work on Google Scholar results: The number and actual results returned by Google Scholar ...
1
vote
2answers
137 views
Different results when experiment repeated
Have you all ever had different results when repeating your own experiment? I did some work for my dissertation and am now redoing it for better images and hopefully for publication. The results are ...
3
votes
2answers
114 views
Should I make my code and data avaiable online when the article is not accepted yet?
We have received a set of reviews for an article that we had submitted to a journal. The decision by the journal editor is a Major Revision. That means, there is a possibility that the article could ...
3
votes
1answer
254 views
Asking name to be taken off a paper
I am asking about the etiquette for getting my name taken off a paper. In grad school I did a collaboration with a PI and the results were limited but okay and I used a very specific microscope to ...
2
votes
1answer
47 views
Can you upload an appendix externally for referencing?
I am submitting an article to an IEEE transactions journal with a strict page limit. We have already scrapped heaps of text, but can't shorten the appendix (or do without) since this would prevent ...
2
votes
1answer
108 views
What can be done (by the community as a whole) to make it easy (for persons other than the authors) to reproduce numerical results in a paper?
For example see reproduced papers repository.
I am restricting the scope to theory papers that have some results obtained through numerical techniques. The fields may be as diverse as physics, ...
1
vote
2answers
51 views
Reproducible, open, transparent, ethical Research when confronted to (patient) data anonymization : is it at all possible in practice?
A lot of Hype goes in reproducible, open and transparency for data analysis theses days in some fields mores that other. The code is to be share as well as the dataset.
How to achieve this in a ...
-1
votes
1answer
87 views
History behind reproducible research tools and literate programming [closed]
Reproducibility was talked about by Robert Boyle in 17th century. Literate Programming was invented in 1984 by Donald Knuth. Reproducible Research tools like R markdown were invented recently. My ...
5
votes
3answers
236 views
Is it acceptable practice to make the source code of a re-implemented paper available online
It is the usual practice that researchers re-implement others' work for the purpose of comparison or result verification. If for instance researcher A re-implements researcher B's paper and improves ...
60
votes
6answers
12k views
How to proceed when the baseline (state-of-the-art) published results claim much better performance than I can reproduce?
I am graduate student, to finish my degree I need to build methods outperform what is already there. An issue that I came across with, is that two papers reported way (I mean more than 20%) more than ...
4
votes
2answers
207 views
How to deal with non-reproducible research?
For context, I am a PhD student in bioinformatics, working partly in development of new statistical models and partly in applying models to data.
I recently came across a paper which describes an ...
1
vote
1answer
216 views
How do we test in academia, if a research area is a money burning machine or non/pseudo-scientific?
In the recent decades more and more meta studies were undertaken on the reproducibilty rate of distinct research areas like psychology, biomedicine...
The results are probably shocking to most ...
3
votes
1answer
175 views
What are the advantages of writing a thesis with rMarkdown? [closed]
I think the question is self-explanatory but to add a bit of a background, I'll be starting my M.Sc. research quite soon and I want to create and maintain a workflow that allows me to devote my time ...
-1
votes
1answer
121 views
Are most real-world (engineering) problems about scale? [closed]
As an engineering post-graduate student a few years back, and now an engineer working in industry, I have often had this realization very strongly - that most problems we "adults" work on are about "...
3
votes
2answers
114 views
Is it worth posting reproducibility studies
As an undergraduate interested in Academia, is it worth posting your reproducibility study of a paper on arXiv and linking to it on Google scholar? Is it useful to future graduate schools? ...
4
votes
2answers
268 views
Incentives for independent researchers to work on reproducible research
For the sake of high-quality science diffusion, there is no doubt that reproducible research is a good practice as it helps to demonstrate the robustness of the proposed methods and boosts the ...
6
votes
2answers
253 views
Is it beneficial for a student to publish their research code and data?
There are at least four benefits of releasing the code and data produced during a thesis or research project (citing textually this laboratory):
Allows to reproduce figures in the revisions of a ...
6
votes
2answers
338 views
Am I responsible for not reproducibility of collaborators’ result?
Recently, I have ran into some issues about reproducibility of some of my research results which are obtained by our collaborators from different university. In fact, in our current repository there ...
11
votes
2answers
6k views
What is the difference between repeatability, replicability and reproducibility?
I have seen many instances where authors used the term "reproducibility" and "replicability" interchangeably in the social and behavioural sciences. Sometimes, they distinguish between the "...
4
votes
1answer
122 views
Reproducible science: do entities exist requiring Level 3 of TOP standards?
Quite many (if not most of) scientific articles, related to computational sciences, which I have seen, do not share both data and code leading to the figures.
This may have of course good reasons ...
-2
votes
2answers
243 views
How to become a reliable researcher? [closed]
I am a biology student with major interest in neuroscience and molecular biology. It's my first year at university.
What should I do if I want to be a reliable researcher in biological sciences, ...
8
votes
2answers
198 views
Reproducing work of others - name them as conflicts of interest?
We are currently reproducing several papers in a specific sub-field of computer science under different experimental conditions. When we submit the paper, we will be required to declare people who may ...
9
votes
3answers
622 views
What exactly would count as copyright infringement in a replication paper?
I have been following the review process of a replication journal
In particular, it has an open review which can be seen.
On a particular submission they are having a debate about whether copying ...
2
votes
3answers
158 views
Finding reproducible articles
My idea is learning new methods/fields by reproducing someone else results, and I would like to pick nice/good papers to learn from.
If we define reproducibility as starting from the same samples/data,...
5
votes
2answers
275 views
Reproducible and exact calculations vs. approximate and slightly random results in experiment
When experiments are performed and reported in an academic article, I learned that reproducibility is important. I am implementing a system where I stand between the choices to use GPU acceleration to ...
18
votes
3answers
672 views
Do we know what proportion of scientific reproducibility issues are fraud-related?
A few days ago there was an article on the LSE blog about scientific reproducibility which made little sense to me until I realised they were equating 'low reproducibility' with 'scientific fraud':
...
4
votes
1answer
355 views
Where can I find examples of research misconduct reports?
I am writing a report documenting evidence for research misconduct. The report is to be submitted to the ethics board of a university. Are there publicly available examples of such reports? Not the ...
4
votes
1answer
314 views
What is the best philosophy and practice to use smartphones for academic life? [closed]
I am doing research in theoretical physics, and have learned that smartphones can bring in a lot of distractions to your life if you want to focus on your research and study. Like facebook, twitter ...
5
votes
1answer
157 views
Ask the author of a paper for suggestions of where to publish replication results
I based my thesis on a published paper and ask its original author if he was interested in seeing the results. After approaching him (with your help provided: How to address an author showing ...
84
votes
12answers
12k views
Is it ethical to use proprietary (closed-source) software for scientific computation?
I have data from an ordinary scientific experiment. Now, in order to analyse the data, the most convenient way for me is to use some proprietary, closed-source software.
Is it ethical to use such ...
6
votes
2answers
285 views
How to address an author showing replicated results from his paper?
I have replicated a method of interpolation recently proposed by certain author, and I want to tell him that I have done so. I suppose the author is interested in knowing that students are engaged ...
1
vote
2answers
223 views
How much should we trust that the approach described in a paper solves the problem that it addresses as well as it claims to?
To be honest, I'm not sure how exactly to word my question in order not to cause any miss-understanding.
During my literature review sometimes I encounter with researches which propose an approach ...
63
votes
5answers
8k views
Why not make students reproduce work?
Typically, PhD theses are expected to have novel results. As science advances, it is becoming increasingly difficult to accumulate enough knowledge to begin a project that pushes the boundaries. ...
29
votes
8answers
5k views
Who is supposed to repeat experiments?
A follow up to this question.
There is an issue in science with experiments not being repeated. I asked if PhDs can be done on repeating experiments and the answer is kind of mixed. Most PhDs aren't ...
2
votes
1answer
69 views
Do measurement protocols belong anywhere in the scientific publications?
My research as a starting PhD student currently consists of reproducing and building upon the experiments carried out by other research groups 5--10 years ago. To this end, I have developed a number ...
16
votes
3answers
1k views
Can you do a PhD solely on repeating experiments of others?
In here, they say there is a problem with results not being replicated. I know a PhD is supposed to be "original" research. But could you do a PhD solely on replicating the results of others? ...
27
votes
4answers
4k views
Why are papers accepted even if they don't release code or data to allow reproducibility?
Recently, we had a lecture about Reproducible research on of the slides was:
Reproducibility: start from the same samples/data, use the same methods, get the same results.
Replicability: ...
1
vote
1answer
92 views
How to identify studies that should be replicated?
In psychology voting on which studies should be replicated is established on a website. The ReplicationWiki (that I founded) offers a voting option for studies in economics and related fields, but it ...
12
votes
2answers
581 views
How do you validate a scientific paper that uses a non-public cell/animal line?
Recently my lab published a paper that uses a mouse line from another lab.
Without going into too much detail since not everyone here is a Biologist, this other lab has a mouse with a reporter gene ...
18
votes
5answers
4k views
Can I trust a paper that hides implementation details?
In Computer Science, sometimes we're assigned to reproduce the results presented by a paper that already have been published. In many cases (most of them) the code is not available, making the task ...
16
votes
2answers
526 views
I've reproduced a paper: where shall I share the code and findings?
The majority of papers don't share their code. However, I sometimes want to reproduce the results, and therefore re-code the experiments to try to reproduce the papers.
Where shall I share the code, ...
5
votes
2answers
1k views
When replicating results, what should I do when the author didn't use the method the paper said they did?
I am replicating a model to report on how well it does on a new evaluation metric I am proposing. The method/hyperparameters of the model are given in a paper about it, where it breaks all benchmarks ...
3
votes
2answers
190 views
How can we perpetuate reproducibility?
Two years ago, Roger Peng discussed in his blog the ramifications of requiring reproducibility for the first producers of research objects, but not for reusers. He then proposed
What I think a good ...
11
votes
5answers
304 views
Are virtual machines good for reproducibility and open science?
Virtual machines (VMs) provide a way for scientists to package not only scientific software, but also data, external dependences, and even entire operating system configurations, facilitating a ...
5
votes
4answers
549 views
Ethical considerations in publishing raw data
Apart from taking measures to maintain confidentiality, do I need my ethical approval to specify permission to publish raw data along with my study? I'm trying to facilitate reproducibility in my ...
16
votes
3answers
2k views
Open Source Software and Reproducibility in Computer Science
As a PhD researcher in computer science, my scientific results often take the form of algorithms, which in turn come with implementations. I am convinced that releasing these implementations under an ...
27
votes
5answers
5k views
What to do when the claims in a paper are proven to be wrong?
What can be done to correct an error that might be hiding a certain author's sole purpose of publishing a paper, no matter the soundness of their method?
In a nutshell, the particular case that ...
13
votes
3answers
319 views
Where do you place the link to open data in a manuscript?
Assume you are writing a manuscript and you have uploaded your raw data to a relevant hosting site (e.g., The Dataverse Network, FigShare, etc.). Where in the manuscript should you indicate a link to ...
