I'm comparing my method against existing methods by doing some experiments on one dataset. The existing methods do not provide results on the considered dataset. My question is: Can I use default parameters from the existing methods? The fact is that there are plenty of existing methods and it will take a lot of time to fine-tune for each existing method. Thank you.
closed as off-topic by corey979, einpoklum, user3209815, Scientist, Jon Custer Jul 2 at 16:21
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is not within the scope of this site as defined in the help center. Our scope particularly excludes the content of research, education outside of a university setting, and undergraduate admissions, life, and culture." – corey979, einpoklum, user3209815, Scientist, Jon Custer
Can I use default parameters from the existing methods?
Yes, but should you? Default parameters are unlikely to provide optimal performance, so you may like to try various parameters.
In my view this depends on a number of things:
a) Are the default parameters of the competing methods advertised as being widely applicable?
b) Have authors of competing methods provided appropriate guidance for how to tune?
c) Does what you are proposing require a lot of tuning, and did you tune yours in order to beat the others?
Answers a) yes b) no c) no point in the direction that you can use other methods with default tuning. If the resulting score is 3:0 or 0:3, my advice is clear. If it's 1:2 or 2:1, well, one needs to look at details, for example, is provided guidance by others so weak that chances are 95% of the users will use the default tuning anyway etc.?
Assuming that your goal is to publish a paper about your method, a way to answer your question is to ask yourself: given clear indications about how your method is evaluated and how the other methods are evaluated, would a reviewer consider this comparison inadequate, sufficient or convincing?