I am working on my MPhys thesis, on a certain dataset. Part of this dataset has been analyzed by my current supervisor during his PhD, with published results. He asked me to expand on that work by completing the analysis on the whole dataset.
When I repeat the analysis on the part he had already worked on, I get different results. Unfortunately he has not published any detail about how exactly he tackled the problem, and instead relied on his memory. He gave me advice like: put this parameter to 50, use feature A, use feature B, disable feature C and so on.
I was able to reproduce his result to within 10%, but yesterday I was horrified when I discovered that this was done without using one of the features he recommended, let's say feature A. I have played with the data since and am totally unable to reproduce the result (it is off by a factor of 2 when using my supervisor specifications).
Using feature A totally makes sense, so I am not sure on what to do: I am afraid that should I tell him about this problem, he would delay my upcoming viva. This has a lot of unwanted consequences, for example would force me to pay extra money to the University