6

Recently I've implemented a pipeline given from a paper, there are three main steps two of these are offline and the third one is real time. The real time part was particularly difficult to implement, and mainly because of lack of details. More specifically the problem was the authors provided a function and they said "we minimize this function". The problem with that is that is wasn't explained how they actually solved the problem and I had in the first place work out all the mathematical details by myself. I've done a detailed analysis that wasn't provided by the original paper, like predicting solution in some cases, computational complexity. Given that this algorithm was real time then this analysis can be useful for whoever would like to implement it.

The question is, given a paper that tells you "we solved our problem using this and we have obtained these numbers" can be worth publishing all the analysis that can theoretically justify those numbers?

In my mind this could also be useful for validation purpose of whoever wants to implement it.

Also, it would be nice to have some example of this if it happens.

Improve this question
6
  • The original paper cite any patent or restriction?
    – prmottajr
    Commented Mar 22, 2018 at 18:05
  • Not really, the thing is that paper explains what they used for an implementation. They explained what problem they tried to solve (with a formula). But my point is many technical details are missing. It's essentially one of those problems where they give you a mathematical expression of something (which is a minimization problem in my case) and later they say what hardware they used to find the solution, but they don't actually explain how they did set up many things in order to be executed on a computer. It lacks of these bits, that to me made difficult the implementation.
    – user8469759
    Commented Mar 22, 2018 at 18:38
  • At the end, as usual, they provide experimental results. But my point is more "if they'd done a more careful analysis that could have possibly be useful". Like "theoretically with this implementation you can actually achieve more even theoretically".
    – user8469759
    Commented Mar 22, 2018 at 19:07
  • I think that you could publish that (if it is of your interest of course) as An implementation for the solution presented in "Original Paper". Or you can always upload it to GitHub as well. But it is not possible to say that your implementation is exactly the same if they don't expose their code.
    – prmottajr
    Commented Mar 22, 2018 at 19:52
  • It is of my interest, is there any way I can get confirmation?
    – user8469759
    Commented Mar 22, 2018 at 20:11

1 Answer 1

Reset to default
1

This is a problem with a lot of research these days: reproducibility. From my experience publishing an implementation approach to a paper, when no approach was given is a valid academic contribution, as there could be various approaches each with its own advantages and disadvantages. If you can expand your work to include multiple reasonable potential approaches and their outcomes, as well as potential pitfalls that could arise (you probably already went through both of these with your work), it would be a significant worthwhile contribution.

The next question is whether the journal you submit to sees it this way, and that can be a hit or miss. The only way to know is to try. Try to frame your work as "Approaches to X (where X is the sub-problem you tackled), instead of linking it too heavily with the original work (where X is e.g. the name of the original paper). If everything else fails, you can always try a conference or open publication although the "social credit" for these will be much less.

I also support the idea of publishing your code on github. The more we share the better off we all will be. This open sharing approach has aided in the significant machine learning publishing boom, and fast pace of improvements we are seeing.

Improve this answer
1
  • Can you provide an example where this happens? Like Paper A with a poor implementation description, and Paper B that actually implements it?
    – user8469759
    Commented May 18, 2018 at 20:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .