0

There are different metaheuristic algorithms for evaluating optimization benchmark problems, some give better results than others and some don't. If a novel metaheuristic has a very close fitness function value to another algorithm created before, let's call A, but is not better than the fitness value of A, what could be the outcome of peer-review process? Would you accept it or there is a rejection possibility for the paper if you were one of the reviewers? Can you explain it by showing an example if it is affordable?

2 Answers 2

1

It is possible, but unlikely that such a paper would be accepted unless it shows some significant advance other than just the result for this particular application. For example, if the methodology were sufficiently new and different and had applications elsewhere, then a knowledgeable reviewer might recommend acceptance. But the reviewer would probably need significant expertise to judge this.

What you describe doesn't seem to fit that characterization, though there isn't enough detail for an analysis here.

A technique that gives additional insight into a class of problems would qualify, if it is noticed.

So "novel" need to be "novel enough" or "significantly novel" to pass the bar.

3
  • Thank you, what do you mean by having applications elsewhere? can you clarify a bit on it?
    – sherl.lol
    Dec 11, 2022 at 15:40
  • Mostly applications to other similar (or not so similar) problems. In math, for example, a new proof technique might be more valuable than the theorem it was used to prove. A new proof of an old problem might be very novel and valuable. Or not.
    – Buffy
    Dec 11, 2022 at 15:42
  • Thank you, got it.
    – sherl.lol
    Dec 11, 2022 at 15:43
1

It is not possible to reduce the entire scientific value of a work to a single number. Even in industrial settings, there is rarely a universally applicable fitness metric by which all approaches may be evaluated: some may be less accurate, but also less computationally expensive, some may be better suited for a specific subclass of problems, the list goes on.

Some researchers in ML lean towards the leaderboard approach to algorithms and models ("if it is not ranking high, it is useless"). At the same time, if everyone was competing on the same leaderboard, we would have maybe 50 papers a year, not tens of thousands.

So even if your novel metaheuristic does not beat SOTA performance, it still is useful. Beware of low-hanging fruit though: if you fail to take the next step and it is trivial (examples may include combining this approach with an existing one), you will essentially get scooped. Good for the overall scientific progress, not so good for your career.

2
  • So do you think leaning towards the leaderboard approach is considered by researchers or caring about the novelty is a priority for them?
    – sherl.lol
    Dec 11, 2022 at 16:18
  • 1
    It is ultimately on you to explain why your approach is useful. At the end of the day, the venue you will try to present your work in, be it a journal or a conference, will be thinking "why would our readers/attendees find it useful?". Leaderboards are kind of a shortcut, even if the researchers would ultimately see some proposals as an attempt to game the leaderboard itself and not as a bona fide solution to an original problem. But there are many more ways to explain and support usefulness than just "it ranks as #1 at XYZ leaderboard". And it is your job to do so.
    – Lodinn
    Dec 11, 2022 at 16:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .