11

I am currently reviewing Paper A for prestigious conference. Earlier this year, Paper B was published in very prestigious conference.

Paper B was a breakthrough in our sub-sub-field. It proved an “impossibility result” (i.e., it is impossible to come up with an algorithm with certain properties). This proof very heavily relied on numerical minimization (not symbolic). Since this was a low-dimensional minimization, I guess the reviewers did not object to using the black-box minimizer as is. The math community generally disagrees on whether using numerical minimization is an acceptable method of proving bounds, but that’s a discussion for another time.

Paper A is basically saying the work in Paper B is unsound (in a polite way) because it relies on numerical methods, and they provide a new approach with a sixty-page analytical proof that the same impossibility result holds (and now claim they are the first to do so because Paper B is unsound).

I personally looked at Paper B and I am honestly okay with the proof method there. I understand numerical methods are not “rigorous” per se, but the function optimized is so smooth, well behaved, simple, convex, etc. that I am willing to bet any decent optimizer can and will find the correct minimum. I coded the minimization in less than five minutes in Python and found the same minimum as the authors.

I am not sure how to rate the paper. While technically, A is the first to analytically prove the result, the result was already shown in B. I am also not motivated to verify over forty pages of straightforward algebra/differentiation, etc.

What would you do in this case? Just let the program committee decide?

1
  • Answers in comments and tangential discussions have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review this FAQ. Comments that do not request clarification or suggest improvements usually belong as an answer, on Academia Meta, or in Academia Chat. Comments continuing discussion may be removed.
    – Wrzlprmft
    Commented Aug 5, 2023 at 16:15

2 Answers 2

13

I don't understand why you say "that's a discussion for another time". The whole point of paper A seems to be to prove the result in a way that is uncontroversially valid. So there is no way to review it without considering the question of whether the methods of paper B actually constitute a proof.

And in doing so you should be thinking about the viewpoint of the conference, not just your own viewpoint. Would a significant proportion of people who attend this conference be likely to value having a fully rigorous proof? If so then paper A seems to be suitable for this conference. If not, then maybe a math journal would be a better fit.

1
  • 2
    "Would a significant proportion of people who attend this conference be likely to value having a fully rigorous proof?" No. Commented Aug 1, 2023 at 14:43
9

Providing an analytical proof over a numerical one is providing value. Whether this is sufficient for inclusion in conference A, depends on the inclusion criteria. Presumably, the contribution of B to your field was more than numerically minimizing a function (that seems rather trivial). Presumably a significant part of their contribution was constructing/finding the function to be minimized. If the contribution of A consists only of finding the minimum of the same function analytically, that seems like a mimimal advance (but an advance nonetheless).

I also find little motivation to verify over 40 pages of straighforward algebra/differentiation, etc to verify the analytical solution when I personally don’t mind the numerical solution.

Whatever you end up recommending as to the suitability for inclusion, you are still going to have to do this. As a reviewer, you are expected to give your opinion on the correctness of the result.

9
  • "Providing an analytical proof over a numerical one is providing value" I am not disputing that. The contribution of B was much more than this result (as you say finding the function and the set up are non-trivial). Optimizing it is just a plus. Paper A just sets up the problem differently and also reaches a function which they optimize, just analytically. "Whatever you end up recommending as to the suitability for inclusion, you are still going to have to do this": No, I do not have to do this. The conference asks us to read at most 12 pages (and more if we would like, but optional). Commented Aug 1, 2023 at 14:47
  • 6
    @Anyon "can you really make an informed recommendation based only on the main text" No, and that is precisely the reason why you do not submit 60 page papers to conferences, you send that to journals where someone will spend a year to review your paper and vet it carefully, not verify every detail in 2 months. Hence the "read 12 pages, and anything more is extra" Commented Aug 1, 2023 at 21:15
  • 2
    @user3508551 Exactly, I would expect the full version to go to a journal and the authors should make an abridged version with perhaps an outline of the proof strategy if they want to submit to a conference also. If the whole point of the paper is to give a fully correct proof, then sending it to a venue that doesn't check correctness makes no sense. Commented Aug 2, 2023 at 8:02
  • 6
    And even for a journal, I wouldn't check correctness unless I believed that it mattered. If the paper is clearly not interesting enough for the journal, then everyone will be better off if you just say that and let the authors get on with submitting elsewhere, rather than cause several months delay while you go through all the details. Commented Aug 2, 2023 at 8:06
  • 1
    "Whatever you end up recommending as to the suitability for inclusion, you are still going to have to do this." I don't understand why (and know enough people who definitely don't do that). If it's clear from the outset that the paper would not be published (wrong topic, "even if true wouldn't clear the threshold for publication", etc.) why should you waste time going through the details if it won't change the result at all?
    – Voo
    Commented Aug 2, 2023 at 9:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .