In my master's thesis work, I am using several algorithmic measures to assess the difference between conditions.
One measure in particular is important to predict 'quality'. This may sound weird in abstraction, but there is a standard written by a number of related institutions in my field, and a reference implementation is given as the standard.
During course work, I was given an implementation of the measure which is written in a different programming language. It is labeled as meas_author, where meas is the name of the measure and author is the author of a well-known book in my field. The book also contains measure implementations that are often cited, in the same programming language as the one I received. I thought the versions were identical.
Now I have found a difference in outputs that is non-trivial (meaning that depending on the implementation, the outcome leads to different results and interpretation). After comparing the code, i noticed small comments like
JSCHWARZ-04-04-2012:changed something for speed
and that the file structure is different.
Personally, this is a Problem because
- The coursework I did based on the modified implementation has the same results as a paper from someone with comments in the code, suggesting both are not comparable with results obtained from standard implementations (!)
- The code may have been used in other coursework, theses, or peer-reviewed publications (some of which received awards) from the people involved or using the measure
- I need to decide whether or not to keep using the measure
- I need to talk about this with my thesis supervisor at some point, especially about the flawed course work
Question: How do I handle this issue? I don't want to directly escalate it as I of course could be wrong. I could talk to the person supplying the code first, or to my supervisor. The former would be least drastic, but I'm not sure what I would do if he didn't cooperate/agree this is a problem.