8

I'm currently writing my Master's thesis in machine learning.

My understanding of academic papers are that you describe what you did in methodology section and what the outcome was in the results section.

But my problem is that the research was very iterative, meaning that the results from early experiments made med tweak the parameters and the algorithm until the results finally could tell me something about my original research question.

Should I describe this process in the methodology and show the results of the earlier experiments, or would it be better to only describe what was done in the final experiment? And if the first alternative is better, should I describe the results of the early experiments in the methodology since they shaped the parameters of the later experiments?

  • Perhaps your thesis will conclude some hypothesis is true. Did any two iterations employ the same data (even partially)? I presume you're aware of the importance of avoiding the statistical fallacies of data dredging (and see here.) The methodology section should describe how you avoided each of those (relevant) fallacies, perhaps by stating you never twice employed the same data (thus mentioning the iterations). – MarkDBlackwell May 11 '16 at 14:30
7

I wouldn't outright reject the utility of discussing failed iterations in your thesis. If handled deftly, a discussion of failures could be just as informative as a discussion of your final approach. Far too often, such discussions are omitted from papers owing to lack of space.

Think of it from your reader's point of view. What does s/he stand to gain from reading about failed approaches? If you could provide an analysis of the "nature of problem" and the "ill fitting approach", and put it in a larger perspective, it might help readers avoid repeating your (very tempting) mistakes in their works.

  • Great, thank you. That's what I'm thinking. But would you then put the results of the failed experiments in the methodology to explain why the parameters changes between the experiments? – Andreascmj May 11 '16 at 8:01
  • You need to check with someone who's familiar with your work, but if I were to give a quick suggestion: the methodology section precedes the failed approaches section; every time you describe a parameter, cite the appropriate position in the next section that describes the background. This way, readers are already familiar with the methodology before the read about failed approaches and put them in perspective. A hasty reader might even skip it. – N. CHATURV3DI May 11 '16 at 9:36
  • Thanks. Then I wont go in to too much detail on the early experiments in the methodology, but add a failed approaches section to the results, and explain for each parameter that this parameter looks like this, because without it the results become as in [failed approaches] – Andreascmj May 11 '16 at 9:48
  • 2
    I agree with this answer, but the OP should set a high bar for "what the reader would learn". Please keep mildly interesting "bad beats" and "debugging" stories out of your research paper, and either talk about them on your blog or when talking about this work face-to-face as a conference. As a rule of thumb, I suggest my students to only include failed attempts if they directly contradict published research or extremely commonplace approaches. – xLeitix May 11 '16 at 10:32
  • Yes, sorry, I should clarify. The failures are not bugs, but iterations of the same experiment that increase how detailed conclusions can be drawn from the results. The first experiment resulted in two clusters with 1 sample in the first, and 100 000 samples in the second. The reason for this was because one parameter affected the clustering too much. The results of that experiment is what made me remove the parameter, which resulted in clusters that could better be analyzed. So my question is, should I describe the first experiment and its results, and if so, where in the paper? – Andreascmj May 11 '16 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.