3

I'm writing a master's thesis about a novel method. It is a multilayered algorithm with very large number of configurable parts and parameters. Furthermore, it involves machine learning and big data, which means each testing run is quite a substantial resource and time-consuming process. That's why researching the effect of every single parameter of this method (or worse, the combinations of possible parameters) would have been an enormous task for a single person with a regular, unrelated job.

Is it acceptable then to omit providing data/results for some less significant parts of the algorithm, and just justify these choices with a more or less vague description of why they seemed promising or "good enough"? Should I mention limited time as a reason for omitting that data?

1
  • 2
    What does your advisor think?
    – Bryan Krause
    Oct 18 '21 at 22:44
5

Omitting anything is rarely a good idea; consciously not doing some of the experiments and giving some reasoning for that is perfectly fine and happens all the time in research.

Citing limited time as a (personal) reason is likewise a terrible idea, IMHO.

Good:

  • With 37 hyperparameters, the search space is enormous. Parameters $\alpha$, $\kappa_1$ and $\kappa_2$ are of special interest, as indicated by prior research (Claus, 2019). In addition, $\lambda$ is pertinent to the problem because of the Moon phase constantly changing. Other parameters were left at their default values as they were deemed less significant (see Table 42 for details).

Bad:

  • I am working a full-time job and don't have the time to tune all 37 parameters so I toyed with just 4 hoping it's good enough.

The difference here, of course, is that in the first case you refer to some objective truth: experiment would take a long while for anyone and at the current state of research, it is more fruitful to focus on a handful of important parameters and only concern yourself with other, less significant ones, later.

1
  • When citing time as a factor, cite the (estimated) times required to perform each part of the analysis to a certain confidence standard, then state how you used those estimated times as objective metrics to decide what to study and what not to study. Oct 19 '21 at 12:55
2

There are a couple of interesting ways of dealing with this, but mainly consider that you need to enclose or delimit your experiment's reach and expectations.

Since most of the hyperparameters are tuned by hand, stating that your p1=0.5 for empirical reasons is perfectly fine.

As @Lodinn mentioned, focusing on those parameters deemed as important is also a good idea. Since some of them are not that impactful, their analysis is not in the scope of this work. You could also consider them as future work :)

Edit: Clarifying: Yes, it is OK to left experimentation settings unexplored, but you need to provide a reasonable explanation on why you decided to do that.

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Oct 19 '21 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.