to finish my degree I need to build methods outperform what is already there
No, that is not true. You need to deliver a piece of proper scientific work and advance knowledge and that does not depend on what direction your findings point.
Of course, things are easier and more pleasant if your implementation is better. But the actual scientific part of your thesis is to study both the old and your approach scientifically and then conclude whether one is better (and possibly in which situations).
The difficulty in your situation is to proove that the discrepancy to literature is not due to your incompetence or lack of hard work (=> you deserve a bad mark) but actually due to "nature" not being as it was supposed to be by the previous paper.
What you can and should report is
- that you were not able to reproduce the findings in papers 1 + 2,
- in consequence have been in communication with the authors.
- Importandly, that your implementation has been confirmed as correct by private communication with the authors of paper 2 and by comparison with (confidential) code you received from the authors of paper 1 again by private communication for that purpose.
If
Well, it turns out they they are not using the data they claim in their the paper, of course their results are different than my reimplementation.
means that you got the data set they actually used and got the same results with that, then you can also report that for a related data set, the same results were obtained.
If not, it may be possible to kindly ask the authors of paper 1 + 2 whether they'd run a data set you send them and give you the results of their implementations so you can compare that to your results. You can then report (hopefully) that equal results were obtained on a different data set and thank the authors of those papers for running your data.
The last two points should make amply clear that the discrepancy is not due to a fault in your implementation - which is what counts for your thesis.
As a personal side note, I got top grade on my Diplom (≈ Master) thesis which (among other findings) found that the software implementation I was using did not work as it was supposed to. I was able to point out a plausible and probable reason for that bug (which may have been a leftover debugging "feature") - which is much harder for you as you don't have access to a running instance of their software that you can test (= study) to form and confirm or dismiss hypotheses about its behaviour.
As an addition to what @Buffy explained already about the possibility of honest mistakes in published papers:
As scientists we tend to work at the edge of what is known. Which also means that we're inherently running a high risk of not (yet) knowing/having realized important conditions and limitations of what we are doing.
We thus also run a comparatively high risk that tentative generalizations we consider may turn out to be not all that general after all. Or that we may be plain wrong and realize this only later (or not at all). I believe it is very hard for humans to be completely aware of the limitations of the conclusions we draw - possibly/probably because our brains are "hardwired" to overfit. (Which also puts us into a bad starting position for avoiding overfitting in e.g. machine learning models we build)
The take-home message from this is that we need to be careful also when reading published papers: we need to keep the possibility of the paper being wrong, containing honest mistakes or not being as directly applicable to our task at hand as we believe at the first glance in mind.
I missed something during the implementation.
I experienced something similar once when I was also implementing a reference method from literature (related but different field). It turned out that different defaults in the preprocessing of the data caused the difference - but only after I had the bright idea of trying out to omit a preprocessing step - although the model doesn't make much sense physically without that step, but the paper didn't mention any such step (neither do many papers in my field who do use that step because it is considered necessary because of physics).
- They are not honest.
While that is of course possible, I've seen sufficient honest mistakes to use Hanlon's razor (which I first met as Murphy's razor): and not assume dishonesty or misconduct unless there are extremely strong indications for that.
Proving superiority may in any case be something that is impossible due to limitations in the old paper.
E.g. if they report validation results based on a small number of cases, the uncertainty on those results may be so large and thus it cannot be excluded that the method is better than it seemed that truly improved methods later on will not be able to demonstrate their superiority in a statistically sound manner.
Still, such a shortcoming of the old paper does not limit the scientific content or advance of your work.
"it turns out they they are not using the data they claim in their the paper"
Can we get some more details about this (or more emphasis of that particular part of the question)? The answers I've seen so far seem to assume that the differences might be due to an honest mistake, but If they published results on one dataset and claimed they were for another, that needs to be reported to the editor.