Skip to main content

Questions tagged [reproducible-research]

Questions related to research in which data and/or code is made publicly available so that others may verify the findings and build upon them.

Filter by
Sorted by
Tagged with
0 votes
0 answers
78 views

Why is it still considered acceptable to publish papers without also sharing the original data and code? [duplicate]

Given the "replication crisis" and numerous examples of outright fraud in academia in the past decades, why do reputable journals still accept papers where the authors don't share 100% of ...
JonathanReez's user avatar
  • 1,558
1 vote
2 answers
109 views

What aspects of software shared with a quantitative paper are the most important?

I'd like to get the software-sharing aspect of my research paper right. (This empirical paper in health economics relies on code for modelling and simulation.) Besides open-sourcing the code on GitHub,...
schrödingcöder's user avatar
0 votes
1 answer
186 views

Consent for Using Scale

I am doing research on identity status. The scale which I intend to use was developed in 1995. I have attempted every possible way to reach out to the corresponding author, but haven't received any ...
Rabab Saeed's user avatar
3 votes
4 answers
271 views

Is it ethical to present the performance of an implementation that uses undisclosed optimizations?

Our research group sometimes does the following: There is some problem P, and we are writing a paper about a novel technique T. In one of the sections of the paper, we try to empirically show that T ...
Grzegorczyk's user avatar
2 votes
1 answer
365 views

How do you reproduce the simulation result of paper you read?

I need to reproduce the simulation done in a research paper I read. I tried to do it by myself last 2 months, but unfortunately without success. I wrote to the first author of this paper but didn't ...
Aid22's user avatar
  • 21
28 votes
6 answers
6k views

Are experiments described in scientific papers *actually* peer-validated today?

Background: the answers and comments on this question prompted this question. I know that when a new scientific theory is published there is a rush for people to devise experiment to prove or disprove ...
LorenzoDonati4Ukraine-OnStrike's user avatar
1 vote
1 answer
218 views

Journal of Reproduction

This question coupled with the general acceptance that a lot of scientific fields have a reproducability crisis has made me wonder whether there should be a specific journal/database that is dedicated ...
ScottishTapWater's user avatar
-1 votes
1 answer
143 views

Does using methods from one research domain to another research domain counts as scientific contribution?

If method A was developed and published in XYZ Conference or Journal and then method A was used to address problem B, can it be regarded as a contribution to the research community? For example, can ...
techno's user avatar
  • 7
8 votes
2 answers
543 views

How much should I tidy up code before I share it?

I'm about to give my first conference presentation, in historical linguistics. (In case it matters: I'm an undergraduate presenting at an undergraduate conference in the UK). My presentation and ...
extremeteenbible's user avatar
31 votes
4 answers
5k views

Is it okay to upload code I wrote for replicating someone else’s simulation study?

I have code I wrote to replicate a simulation study by another author. Is it okay to publicly upload this in my code repository? This will include: code (my work, based on the author’s description in ...
user avatar
12 votes
7 answers
3k views

How does “reproducing other labs' results” work?

I don’t understand where "reproducing results from other labs" fits into the running of a given research lab. (I'm not in academia, and I confess I only consider this function in the context ...
chai's user avatar
  • 123
7 votes
1 answer
349 views

A paper is claiming to calculate something for which data probably does not exist

I am working on a project where the objective is to test if X causes Y. In the field, there are two methods for calculating X. One is considered more accurate, but require data often not available for ...
Martan's user avatar
  • 173
2 votes
2 answers
924 views

How can I find if a certain study has been replicated or the results reproduced?

I am trying to find additional research about a certain topic. I have found one study that is specifically about this topic, and the results are very interesting. I would like to find out if any ...
elmer007's user avatar
  • 129
4 votes
1 answer
628 views

Do academics make decisions like other humans?

There are now 100s of studies from behaviour change sciences that show that humans take mental shortcuts to make decisions. For instance we eat at a busy restaurant, because, as a rule of thumb, they ...
Rob Blackie's user avatar
15 votes
2 answers
3k views

Is it appropriate to re-use data from previous publication for a new study?

Data from a published study was deposited on the Harvard Dataverse abiding by the replication policy of a scientific journal. I have no connections with the scholars who collected the data and ...
000andy8484's user avatar
5 votes
1 answer
554 views

Research on gender and academic misconduct

In professor David Shanks' talk "Does social science tell the truth? (2016)" at 15:14 he claims that there is an apparent gender bias among his examples of people that have committed ...
user avatar
0 votes
1 answer
167 views

Submitting two related papers to two journals simultaneously [duplicate]

I have spent 2+ years working on a project in computational neuroscience that has interest for clinicians and now I am starting to write the paper text. I want to submit it to one of neurology-related ...
demitau's user avatar
  • 1,246
47 votes
9 answers
9k views

Why isn't it the norm to have research repeated immediately by other academics?

The potential benefit to society of good research, and the potential costs of acting on bad research, seem to be extremely high in both directions. Why is it the norm to simply peer review research, ...
stevec's user avatar
  • 1,070
0 votes
0 answers
85 views

A measure implementation from my group deviates from an official implementation

In my master's thesis work, I am using several algorithmic measures to assess the difference between conditions. One measure in particular is important to predict 'quality'. This may sound weird in ...
Jonas Schwarz's user avatar
5 votes
3 answers
1k views

What would you do if the author of a published CS article does not respond to code requests?

Since reproducibility is a major principle or basic norm of scientific experiments, simulations, and computations, what would you do or what to do if the author of a published CS article does not ...
cser's user avatar
  • 69
-2 votes
1 answer
191 views

Why are most article pdfs not self-reproducable? [closed]

Context In light of the perishability of external resources as illustrated here: https://xkcd.com/1909/ I was wondering if it is desirable to ensure a single PDF, including its research code, can be ...
a.t.'s user avatar
  • 115
4 votes
3 answers
2k views

How to answer a reviewer asking for the methodology code of the paper?

Recently I have submitted my paper to IEEE Access Journal which has a binary peer review process, and I got rejected. Still, they allow me to answer the reviewers' concerns and resubmit again. One of ...
Bonnie's user avatar
  • 43
23 votes
6 answers
7k views

Author has published a graph but won't share their results table

I am trying to reproduce some of the results in a paper from 2018 where the results were included in a graph but no exact values were given. There is no results table to go with the graph. I can't ...
Espore's user avatar
  • 354
4 votes
1 answer
303 views

Is there a way to see which papers have used a particular R package?

I'm a new user of an R package (tidy text mining) and I'd like to see what papers have used this package as examples for the kind of things I can do with it. Is there a way to do this?
Gabriella's user avatar
  • 143
1 vote
0 answers
56 views

How do I find the ML model referenced in a specific paper when the authors don't provide it?

I'm trying to locate the architecture of an LSTM neural network as described here, but the authors don't provide a repository for other academics. From arXiv, I see more than a dozen other researchers ...
Eduardo Barrera's user avatar
2 votes
1 answer
186 views

Reproducing experimental data from published work

I am doing theoretical study of a material and in my recent work I have compared my calculated results with experimental results from another published work. During review, the reviewer has mentioned ...
Thomas's user avatar
  • 184
3 votes
1 answer
230 views

Google Scholar Syntax quirks: results depend on order of search strings [closed]

I am working on a systematic literature review and noticed something quite disturbing since I want to base my work on Google Scholar results: The number and actual results returned by Google Scholar ...
stefanct's user avatar
  • 139
1 vote
2 answers
1k views

Different results when experiment repeated

Have you all ever had different results when repeating your own experiment? I did some work for my dissertation and am now redoing it for better images and hopefully for publication. The results are ...
R. McDowell's user avatar
3 votes
2 answers
198 views

Should I make my code and data avaiable online when the article is not accepted yet?

We have received a set of reviews for an article that we had submitted to a journal. The decision by the journal editor is a Major Revision. That means, there is a possibility that the article could ...
Coder's user avatar
  • 12.7k
3 votes
1 answer
565 views

Asking name to be taken off a paper

I am asking about the etiquette for getting my name taken off a paper. In grad school I did a collaboration with a PI and the results were limited but okay and I used a very specific microscope to ...
R. McDowell's user avatar
2 votes
1 answer
145 views

Can you upload an appendix externally for referencing?

I am submitting an article to an IEEE transactions journal with a strict page limit. We have already scrapped heaps of text, but can't shorten the appendix (or do without) since this would prevent ...
T. van der Heijden's user avatar
2 votes
1 answer
137 views

What can be done (by the community as a whole) to make it easy (for persons other than the authors) to reproduce numerical results in a paper?

For example see reproduced papers repository. I am restricting the scope to theory papers that have some results obtained through numerical techniques. The fields may be as diverse as physics, ...
Tejas Shetty's user avatar
1 vote
2 answers
68 views

Reproducible, open, transparent, ethical Research when confronted to (patient) data anonymization : is it at all possible in practice?

A lot of Hype goes in reproducible, open and transparency for data analysis theses days in some fields mores that other. The code is to be share as well as the dataset. How to achieve this in a ...
Samusz's user avatar
  • 53
-1 votes
1 answer
137 views

History behind reproducible research tools and literate programming [closed]

Reproducibility was talked about by Robert Boyle in 17th century. Literate Programming was invented in 1984 by Donald Knuth. Reproducible Research tools like R markdown were invented recently. My ...
user2338823's user avatar
5 votes
3 answers
510 views

Is it acceptable practice to make the source code of a re-implemented paper available online

It is the usual practice that researchers re-implement others' work for the purpose of comparison or result verification. If for instance researcher A re-implements researcher B's paper and improves ...
Abdulhameed's user avatar
  • 1,934
62 votes
6 answers
13k views

How to proceed when the baseline (state-of-the-art) published results claim much better performance than I can reproduce?

I am graduate student, to finish my degree I need to build methods outperform what is already there. An issue that I came across with, is that two papers reported way (I mean more than 20%) more than ...
Monii_80's user avatar
  • 521
5 votes
2 answers
599 views

How to deal with non-reproducible research?

For context, I am a PhD student in bioinformatics, working partly in development of new statistical models and partly in applying models to data. I recently came across a paper which describes an ...
alan ocallaghan's user avatar
1 vote
1 answer
298 views

How do we test in academia, if a research area is a money burning machine or non/pseudo-scientific?

In the recent decades more and more meta studies were undertaken on the reproducibilty rate of distinct research areas like psychology, biomedicine... The results are probably shocking to most ...
user48953094's user avatar
  • 4,346
3 votes
1 answer
286 views

What are the advantages of writing a thesis with rMarkdown? [closed]

I think the question is self-explanatory but to add a bit of a background, I'll be starting my M.Sc. research quite soon and I want to create and maintain a workflow that allows me to devote my time ...
Jxson99's user avatar
  • 131
-1 votes
1 answer
174 views

Are most real-world (engineering) problems about scale? [closed]

As an engineering post-graduate student a few years back, and now an engineer working in industry, I have often had this realization very strongly - that most problems we "adults" work on are about "...
shivams's user avatar
  • 119
3 votes
2 answers
136 views

Is it worth posting reproducibility studies

As an undergraduate interested in Academia, is it worth posting your reproducibility study of a paper on arXiv and linking to it on Google scholar? Is it useful to future graduate schools? ...
midawn98's user avatar
  • 213
4 votes
2 answers
359 views

Incentives for independent researchers to work on reproducible research

For the sake of high-quality science diffusion, there is no doubt that reproducible research is a good practice as it helps to demonstrate the robustness of the proposed methods and boosts the ...
JMFS's user avatar
  • 301
6 votes
2 answers
314 views

Is it beneficial for a student to publish their research code and data?

There are at least four benefits of releasing the code and data produced during a thesis or research project (citing textually this laboratory): Allows to reproduce figures in the revisions of a ...
JMFS's user avatar
  • 301
6 votes
2 answers
480 views

Am I responsible for not reproducibility of collaborators’ result?

Recently, I have ran into some issues about reproducibility of some of my research results which are obtained by our collaborators from different university. In fact, in our current repository there ...
Mithridates the Great's user avatar
13 votes
2 answers
11k views

What is the difference between repeatability, replicability and reproducibility?

I have seen many instances where authors used the term "reproducibility" and "replicability" interchangeably in the social and behavioural sciences. Sometimes, they distinguish between the "...
non-numeric_argument's user avatar
4 votes
1 answer
154 views

Reproducible science: do entities exist requiring Level 3 of TOP standards?

Quite many (if not most of) scientific articles, related to computational sciences, which I have seen, do not share both data and code leading to the figures. This may have of course good reasons ...
J. Doe's user avatar
  • 183
-2 votes
2 answers
266 views

How to become a reliable researcher? [closed]

I am a biology student with major interest in neuroscience and molecular biology. It's my first year at university. What should I do if I want to be a reliable researcher in biological sciences, ...
L.Diago's user avatar
  • 105
8 votes
2 answers
241 views

Reproducing work of others - name them as conflicts of interest?

We are currently reproducing several papers in a specific sub-field of computer science under different experimental conditions. When we submit the paper, we will be required to declare people who may ...
malexmave's user avatar
  • 1,226
9 votes
3 answers
859 views

What exactly would count as copyright infringement in a replication paper?

I have been following the review process of a replication journal In particular, it has an open review which can be seen. On a particular submission they are having a debate about whether copying ...
stochastic_random's user avatar
2 votes
3 answers
189 views

Finding reproducible articles

My idea is learning new methods/fields by reproducing someone else results, and I would like to pick nice/good papers to learn from. If we define reproducibility as starting from the same samples/data,...
llrs's user avatar
  • 1,182