When I write a paper comparing my algorithm to someone else's, how should I make sure that the results are not skewed because I implemented the other algorithm poorly? Is it acceptable to write that the other algorithm was implemented to my best knowledge?

  • 4
    If you implement theirs badly - how can you then say yours is good in comparison? – Solar Mike Apr 3 '18 at 10:11
  • 2
    When you implement an algorithm from another paper, that paper usually presents some results, too. If your implementation can reproduce them, it's most likely your implementation is correct. If not - you made a mistake somewhere. – user68958 Apr 3 '18 at 10:13
  • @Solar Mike This is exactly my question. – user2286759 Apr 4 '18 at 13:42

It is usually sufficient to say that you tried to optimize both your algorithm and the baseline(s) as much as you could (provided you actually did so).

Before writing your own implementations though, you should first check that the authors of the baseline(s) haven't published their source code. You may even write to them. I've heard all kinds of replies to such requests: from people sending me the code within minutes, to people telling me their code was lost in an old laptop long ago (the paper was 2 years old...). If they don't have the code, you can ask them how they would/did implement their algorithm.

It would be good practice and very beneficial for the community if you then publish both the code for your algorithm AND (if you implemented them) the code for the baselines.

|improve this answer|||||

You could compare the computational complexity of each algorithm, without considering any implementation. (Of course, comparisons between implementations are interesting, but there are many barriers to conducting such comparisons, e.g., comparisons aren't skewed because of coding issues.)

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.