4

I am physicist and my research is on Physical/Computational Biology. As most of my colleagues, I am used to visit ArXiv daily to check new articles on my field. Some weeks a go I found a really interesting article with a new method that I would like to implement together with a model I am working with.

At first glance I noted that the parameters used in the article were something strange, but I thought it could be due to the different spatial discretisation employed. However, I changed all my parameters to fit with their method and concluded that they are completely non-sense. Even the most simple case proposed in the article is impossible to achieve using their parametrisation (I was able to reproduce using another set of parameters).

My question is: Have you faced something similar with papers in pre-print servers? Is this something people are used to do in order to avoid other researchers to reproduce their work and advance before the article is published in a journal?

Improve this question
2
  • 8
    I don't work extensively with pre-print servers, but I would frown on anyone who writes a purposedly misleading paper using such an excuse, in particular because in all fields that I am aware, arXiV does guarantee originality. I would first assume that the mistake is due to a typo. This kind of error would be very easy to uncover when attempting to reproduce a paper, which is exactly what you did. I would inform the authors about their typo.
    – FBolst
    Commented Dec 29, 2017 at 15:33
  • 4
    @FBolst I assure you, it was not a typo. The whole set of parameters is wrong. I agree with you that if they did that, it is very strange because of the guarantee of originality given by ArXiv. However if they are that paranoid they should not put their work in a preprint server at all.
    – The Doctor
    Commented Dec 29, 2017 at 15:44

1 Answer 1

Reset to default
10

I would not assume that this was intentionally done to mislead other researchers. If they wanted to keep their paper confidential, they would not upload it to ArXiv with incorrect parameters. As you have already shown, people are able to find a valid set of parameters by theirselves.

Please consider also the following possible reasons:

  • The wrong parameters were accidentally put into the paper. One may have forgotten to update the parameters in the paper and left in some placeholders.
  • You overlooked some important fact, e.g., scaling of parameters. So the error might also be on your side or the corresponding section of the paper is difficult to understand.
  • The parameters might only work under certain conditions, thus, the authors might be at error.

However, I would recommend to contact the authors using a short and friendly email. You should not assume an error on their side from the beginning as this might come across as rude. You may rather formulate this as a question, e.g., whether they have an idea why you are not able to reproduce the results. You might either be able to see your own error or help the authors with improving their paper.

Improve this answer
3
  • I like the placeholder theory. It's not a scaling problem, this was my first thought and I tested it. I would never accuse someone, I was just puzzled. I won't contact the authors because I've already invested time parametrising the model myself and the authors won't need my feedback, since the paper will pass through peer-review.
    – The Doctor
    Commented Dec 31, 2017 at 0:26
  • 6
    I think it'd be good to contact the authors anyway. Best case scenario the authors might even be able to correct things before the paper goes out for review. That's assuming there is a mistake - otherwise you might learn something useful for your own research. Worst case scenario if you don't contact them is that neither reviewers or authors spot the issue at all. Plus it's always nice to hear from people interested in one's preprint.
    – Anyon
    Commented Dec 31, 2017 at 3:35
  • 3
    I agree with Anyon. Unfortunately, I have seen several papers with obvious errors that were not detected in peer review. Most reviewers do not have the time and ability to rerun all of the experiments, thus, I highly doubt that incorrect paramters will be noticed at all.
    – J-Kun
    Commented Dec 31, 2017 at 8:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .