2

I need to reproduce the simulation done in a research paper I read. I tried to do it by myself last 2 months, but unfortunately without success. I wrote to the first author of this paper but didn't get any reply still. I am afraid he will ignore my email. I am sure he receives a ton of such emails as mine. :( One of my friends suggested me to use help someone external with reproducing the simulation result of the paper: a freelance who will simulate it for me.

I have never used a freelancer service for my research and do not know if it is a suitable solution in my case.

have you ever had such an experience? Do you use freelancers for helping you with simulations?

Field: computer engineering, satellite communication

Improve this question

1 Answer 1

Reset to default
4

Reproducibility in science is in crisis in several disciplines. Psychology is one, and while there has been no study out to my knowledge, computer science is another I suspect.

If you tried to reproduce the results for two months and was unable to, there's a real chance that they simply don't work, or work under a very specific set of circumstances.

I argue that if you only need to reproduce another work’s simulation to compare with your own work, you are under no obligation to reproduce it if the authors have not provided the public with the tools to do so.

It is perfectly acceptable to “shame” them and state something like “while X’s work is a natural point of comparison, the authors did not provide publicly available code or data that supports their findings. Thus, we were unable to benchmark against their results”.

If you need the simulation results, and have the funding to hire someone to reproduce them - then absolutely hire someone to help. I would however caution you that there is a real chance that you won't be able to do it since the results are simply not reproducible.

Improve this answer
9
  • If it were as easy as "Here is the paper, turn that into code please" OP would probably have been able to do it. That's why I would think a hired low-cost freelancer will not be able to help OP with anything. You could get another CS grad student/knowledgeable person to help you with reproduction and then add them as a co-author.
    – til_b
    Commented Jun 13, 2023 at 14:49
  • 1
    And +1 for shaming the code non-availability!
    – til_b
    Commented Jun 13, 2023 at 14:50
  • 1
    I’ve been involved in some code development projects, so I have some idea how they work. If you’re not offering people a way of reproducing your results, then you’re not doing good science in my opinion. I’m assuming OP is competent, and if so the authors did not offer enough information to reproduce their work - not OP’s problem. Saying they didn’t is the simple truth.
    – Spark
    Commented Jun 14, 2023 at 0:06
  • 1
    @yarchik I'm quite surprised by your surprise about sharing code in research. Perhaps you think more about the context of industry where company trade secret is more of a big deal? In academia, the majority view now is to push for more open sharing of code. So at the very least, even if you disagree, it shouldn't have come up as a surprise for you that some people hold the view of openness (sharing code). At least that's how it is in computer science field.
    – justhalf
    Commented Jun 15, 2023 at 6:55
  • 1
    My answer doesn’t say “share all code always”. It says you cannot have it both ways. If you choose to not share your code for whatever reason, don’t expect people to trust your results or replicate them for you.
    – Spark
    Commented Jun 15, 2023 at 23:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .