4

I am writing a paper which utilizes machine learning to guide clinical decisions regarding a certain disease. This paper will be submitted to some neonatology journals in the next couple months, but I have never written anything about machine learning in this context before. Additionally, if published this would be my first paper ever. Unfortunately, I don't have anyone of whom I can ask kinds of questions.

Currently, the technical portion of the paper includes everything we tried to select a good machine learning model. For example, all hyperparameters are listed, all attempted feature engineering is listed, all algorithms implemented are listed, etc.

A few questions:

  1. Should I include every model we tried including those that we didn't?
  2. Should I include hyperparameters for gridsearch or only the best hyperparameters?
  3. Should I include every method of feature engineering we implemented?
  4. Should I include examples of the data's structure?

General advice is appreciated too.

  • 2
    What do other papers written in your field do? – Bryan Krause Sep 3 '19 at 21:41
  • @BryanKrause I can see papers on google scholar that give a large amount of detail and not enough for replication of the paper. Some even redefine well-known algorthms in their paper. I am rather uncertain as to how properly to proceed. – Joe B Sep 3 '19 at 21:56
  • So who is “we”? if your advisor then ask them... – Solar Mike Sep 3 '19 at 23:27
  • @SolarMike I mean my company and I. None of them are data science people and I've still got lots to learn so we are in a small pickle. – Joe B Sep 3 '19 at 23:29
2

The following is the standard for deep learning literature as far as I am aware:

Should I include every model we tried including those that we didn't?

No, only include your best model (or at least, the ones you want to show). Journal space is limited, so people generally don't include unsuccessful attempts.

Should I include hyperparameters for gridsearch or only the best hyperparameters?

Only the best hyperparameters. If one of your model's selling points is it's resilience to different hyperparameters then you might include a diagram showing how the different hyperparameters affect the results. If not then the worse-performing hyperparameters are irrelevant.

Should I include every method of feature engineering we implemented?

Yes, this is important for reproduction.

Should I include examples of the data's structure?

That would be nice, but perhaps not necessary. Perhaps include it if your data is very non-standard.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.