3

I am working on improving an existing deep learning model which was published in last year A* conference. The code and datasets are made available on GitHub. When I ran them the accuracy was less than what they cited on paper I ran code 10 times answer changed a little(>1%) but the difference of cited score and actual scores are like( approx 5%). what should I report in my paper actual one or one I am getting

P.S: I tried contacting all the authors and also opened an issue on github repository of code

2

The first step when there is a discrepancy between the published work and attempts to reproduce the work is generally to email the authors. Let them know what is happening. Maybe there is a bug in the shared code or something about the initial conditions you are missing. Hopefully they will be able to help you resolve the issue. If not, you should probably publish both numbers and a statement that you could not reproduce their values.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.