Recently I've implemented a pipeline given from a paper, there are three main steps two of these are offline and the third one is real time. The real time part was particularly difficult to implement, and mainly because of lack of details. More specifically the problem was the authors provided a function and they said "we minimize this function". The problem with that is that is wasn't explained how they actually solved the problem and I had in the first place work out all the mathematical details by myself. I've done a detailed analysis that wasn't provided by the original paper, like predicting solution in some cases, computational complexity. Given that this algorithm was real time then this analysis can be useful for whoever would like to implement it.

The question is, given a paper that tells you "we solved our problem using this and we have obtained these numbers" can be worth publishing all the analysis that can theoretically justify those numbers?

In my mind this could also be useful for validation purpose of whoever wants to implement it.

Also, it would be nice to have some example of this if it happens.

  • The original paper cite any patent or restriction?
    – prmottajr
    Mar 22 '18 at 18:05
  • Not really, the thing is that paper explains what they used for an implementation. They explained what problem they tried to solve (with a formula). But my point is many technical details are missing. It's essentially one of those problems where they give you a mathematical expression of something (which is a minimization problem in my case) and later they say what hardware they used to find the solution, but they don't actually explain how they did set up many things in order to be executed on a computer. It lacks of these bits, that to me made difficult the implementation. Mar 22 '18 at 18:38
  • At the end, as usual, they provide experimental results. But my point is more "if they'd done a more careful analysis that could have possibly be useful". Like "theoretically with this implementation you can actually achieve more even theoretically". Mar 22 '18 at 19:07
  • I think that you could publish that (if it is of your interest of course) as An implementation for the solution presented in "Original Paper". Or you can always upload it to GitHub as well. But it is not possible to say that your implementation is exactly the same if they don't expose their code.
    – prmottajr
    Mar 22 '18 at 19:52
  • It is of my interest, is there any way I can get confirmation? Mar 22 '18 at 20:11

This is a problem with a lot of research these days: reproducibility. From my experience publishing an implementation approach to a paper, when no approach was given is a valid academic contribution, as there could be various approaches each with its own advantages and disadvantages. If you can expand your work to include multiple reasonable potential approaches and their outcomes, as well as potential pitfalls that could arise (you probably already went through both of these with your work), it would be a significant worthwhile contribution.

The next question is whether the journal you submit to sees it this way, and that can be a hit or miss. The only way to know is to try. Try to frame your work as "Approaches to X (where X is the sub-problem you tackled), instead of linking it too heavily with the original work (where X is e.g. the name of the original paper). If everything else fails, you can always try a conference or open publication although the "social credit" for these will be much less.

I also support the idea of publishing your code on github. The more we share the better off we all will be. This open sharing approach has aided in the significant machine learning publishing boom, and fast pace of improvements we are seeing.

  • Can you provide an example where this happens? Like Paper A with a poor implementation description, and Paper B that actually implements it? May 18 '18 at 20:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.