3

I am working on improving an existing deep learning model which was published in last year A* conference. The code and datasets are made available on GitHub. When I ran them the accuracy was less than what they cited on paper I ran code 10 times answer changed a little(>1%) but the difference of cited score and actual scores are like( approx 5%). what should I report in my paper actual one or one I am getting

P.S: I tried contacting all the authors and also opened an issue on github repository of code

Improve this question
1
  • 1
    You should probably report both.
    – JRN
    Commented Jun 30, 2019 at 10:23

1 Answer 1

Reset to default
3

The first step when there is a discrepancy between the published work and attempts to reproduce the work is generally to email the authors. Let them know what is happening. Maybe there is a bug in the shared code or something about the initial conditions you are missing. Hopefully they will be able to help you resolve the issue. If not, you should probably publish both numbers and a statement that you could not reproduce their values.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .