I plan to submit an article to a top tier conference in the field of computer science. However, I have certain doubts about the right procedure to compare with state-of-the-art methods in my field.
I want to claim that the approach we propose is better than the state-of-the-art method. To prove this, I have re-implemented the state-of-art and tested it on a benchmarked dataset of my own creation. The method of our proposal seems to outperform the state-of-the-art.
However, the authors of the state-of-the-art paper had neither open-sourced their code nor tested it on a publicly available, benchmarked dataset. So, it is proving to be pretty much impossible to confidently claim that our re-implementation of the state-of-the-art is an accurate replica of what was proposed/implemented in the original paper.
How do I go about highlighting this dilemma in the article I am about to submit, without sounding too vague? I am ready to share a detailed pseudo-code of the re-implementation as supplementary material to the submission. I am also about to make the benchmarked dataset (that we created) public.
However, I would like to ask: what is the standard procedure for handling such doubts? (I am sure I am not the first author to face such a confusion).
