6

An article claiming to solve a great long-standing mathematical open problem was published in a conference proceedings, and was refereed on MathSciNet positively, i.e. the reviewer makes an impression that the proof is valid. However, some details hint that this may not be the case:

  1. the result of this caliber should go to a top-tier journal and get a significant resonance in the mathematical community;

  2. the preprint has been available on arXiv for 15 years before its final publication and underwent eight redactions (with the last redaction about eight years before the publication);

  3. I talked to an expert in this area around 10 years ago and their opinion about one of the earlier drafts of the preprint was that it is erroneous and the author doesn't want to admit his mistakes.

Now some other articles (published in first-tier journals) use this questionable result. And many other people are still working on solving this long-standing problem, and may have doubts about the status of the problem, in view of this publication.

So my question is: how much of the responsibility lies on the MathSciNet reviewer who "validated" the proof? My understanding is that it is their duty to reveal the mistakes in the published article, so that the other authors do not base their work on it. Is there a way to "nudge" the reviewer to make amends to the review?

  • 3
    I think I know which paper you're talking about (although I have not seen the MR review, as I do not have access to MR), but I think (1) this question is too specific to math content and math culture for Academia, and (2) your wording of "Isn't it their duty to reveal the mistakes in the published article" implies that the reviewer was aware of this, and while this might be true, nothing you've said implies this, and thus on face value this seems to be a leading question, and thus probably not suitable (as presently worded) for Academia. – Dave L Renfro May 23 at 20:18
  • @DaveLRenfro Thanks, I changed it into a statement. – mathreader May 23 at 20:32
  • 4
    Peer review is generally conducted without any reward or punishment for the peer reviewer. If you start punishing the reviewers, soon there will be none. – Anonymous Physicist May 24 at 3:49
  • 6
    @AnonymousPhysicist: Technically, MathSciNet reviewers do get a small monetary reward, currently about USD 12.00 per review. While I understand your point, I think it's also important not to conflate review databases like MathSciNet with "peer review" in the usual sense, since there are many differences, and I think confusing the two may underlie some of the mistaken premises in this question. – Nate Eldredge May 24 at 4:10
  • 5
    Related, mathoverflow: Does a referee have to check carefully the proof? and Should I become a Mathscinet reviewer?. But please only say 'reviewer' not 'referee' – smci May 25 at 3:08
13

how much of the responsibility lies on the MathSciNet reviewer who "validated" the proof?

None. The instructions for reviewers don't ask the reviewer to check the validity of the proofs; by a time a paper gets to MathSciNet it has been peer reviewed already. The purpose of a MathSciNet review is to explain what's in the paper and why someone might want to read it.

It is fine to write to contact AMS and ask them to retract a review, if (1) you are a well-known expert in the field, capable of speaking (to a reasonable extent) for the field as a whole; or (2) you are able to conclusively demonstrate that the paper has an error.

If neither of these is the case, while your desire to do something is admirable, realistically there is probably not any effective action available to you.

| improve this answer | |
  • 1
    Thank you for your reply. Maybe I can ask your opinion on the following aspect. Suppose a young researcher obtained a certain small contribution to the problem in some partial case. What would be his or her best course of actions when publishing their result: ignore the article in question whatsoever, or work hard on fulfilling item (2) in your answer ("conclusively demonstrate") and supply the findings in their article? The amount of work to do (2) may turn out to be incomparable with their small but still interesting contribution to the topic. – mathreader May 23 at 23:15
  • 1
    I'd ignore the article in question, if I believed it was wrong. If you have something you'd like to publish, I'd encourage you to seek out the opinion of an expert in the subject of your paper. In particular, if you are a student or postdoc, I'd recommend that you ask your advisor. Good luck! – academic May 23 at 23:36
  • How could one do (2)? Can people who do not sarisfy (1) really write proofs that some paper is wrong to MSN such that MSN actually reads them and works on finding out the truth? – user111388 May 24 at 20:09
18

(Disclosure: I review for MathSciNet.)

how much of the responsibility lies on the MathSciNet reviewer who "validated" the proof?

In my view, very little if any, and by using the word "validate" I think you overstate the case. Reviewing for MathSciNet is not meant to be like refereeing, and reviewers are definitely not asked to check the correctness of a paper as they would for a referee report. The purpose of a MathSciNet review is mainly to summarize what the paper contains, so that prospective readers can quickly tell if the contents of the paper are likely to be of interest to them. I don't think you should take the existence of a MathSciNet review as any sort of "seal of approval" on the correctness of the paper.

The reviewer is certainly allowed to mention errors or shortcomings in the paper if they should happen to observe any, but this is not a requirement or an expectation.

And keep in mind that unlike an referee report, MathSciNet reviews are public to the world and signed with the reviewer's name. If a reviewer says something negative about a paper, especially a very high-profile paper, they might face blowback in a way that an anonymous referee wouldn't. Maybe you think they have an obligation to do it anyway and face the consequences, but it's a significant ask.

Useful reading is the Guide for Reviewers that MathSciNet points reviewers to.

My understanding is that it is their duty to reveal the mistakes in the published article, so that the other authors do not base their work on it.

I would only agree insofar as to say that if the reviewer is actually aware of a specific mistake in the paper, they should mention it. Even so, that would only be out of a general civic duty to the community, not any responsibility explicitly laid on them by MathSciNet. And for me, a vague sense of unease that something is not quite right with the paper would not rise to that level. If the reviewer does not happen to find a specific error, I do not think they have a responsibility to go looking for one. Certainly in my time as a reviewer and in talking to others, I've never had an impression that this was expected.

Is there a way to "nudge" the reviewer to make amends to the review?

Their name is attached to the review, so of course you are free to contact them and say something. But as I mentioned, I think what you're asking is beyond the scope of what's expected of a reviewer. If you think the reviewer actually knows of a specific error and has covered it up, that would be another story, but as I said I don't think they have an obligation to carefully check a paper in hopes of finding errors.

Moreover, MathSciNet doesn't really encourage reviewers to revise their reviews after initial submission. There is no automated system to submit revisions. So even if the reviewer wanted to make changes, they'd have to contact the MathSciNet editorial staff and convince them that changes were warranted.

Now some other articles (published in first-tier journals) use this questionable result.

Well, it's the responsibility of those authors to satisfy themselves of the correctness of results that they rely on. They cannot reasonably take the mere existence of a MathSciNet review as positive evidence of the paper's correctness.

| improve this answer | |
  • 3
    "Their name is attached to the review, so of course you are free to contact them and say something." If you complain about someone else's volunteer reviewing, you are putting your own reputation at risk. – Anonymous Physicist May 24 at 3:51
3

So my question is: how much of the responsibility lies on the MathSciNet reviewer who "validated" the proof? My understanding is that it is their duty to reveal the mistakes in the published article, so that the other authors do not base their work on it. Is there a way to "nudge" the reviewer to make amends to the review?

  1. None. Unlike what your title implies MathSciNet reviews are not referee reports nor are they endorsements. As stated in the guidelines, the reviews are there to help some decide if they wish to read the paper.

  2. Incorrect. The primary duty is to provide context to the paper. Moreover, as noted in the instructions, the review is not meant to start a debate as the original author cannot answer.

  3. This assumes or implies that somehow the review motivated citations to the paper, which would be extraordinary.

If you believe you found an error, you should supply evidence of that error in a comment to the original journal or another journal. This is the most constructive way of correcting a wrong paper. An alternative would be to use PubPeer.

Frankly: why the excitement? People cite wrong results - including papers that have been officially withdrawn by journals - all the time. See this table compiled by Retraction Watch of the Top 10 most highly cited retracted papers. #2 on that table has over 1000 citation since the article was retracted.

| improve this answer | |
0

You state, without qualification, that the paper is erroneous. Actually, that may be true or not. And even if it contains errors, it is also possible that they are immaterial to the main result. You blame it on the reviewer. But the reviewer may simply be mistaken in their analysis - especially if the result is deep and subtle. People make mistakes. Authors do. Reviewers also. It isn't evil intent.

And it certainly isn't the job of a review writer, after publication, to make a claim without evidence that a paper is flawed.

It is the responsibility of the reviewer to try to give a valid report on the paper as best they can analyze it. If they are wrong, they have made a mistake, but I doubt that any reviewer would "cover up" for an author, stating that something was true, when they knew it was not.

But it is also the responsibility of the reviewer to include in any report that they can't follow the argument to its end, if that is the case. But that is about all that they can do and all you can ask for.

But, the proper response to an erroneous paper is to publish a correct one. That can be done by anyone. If you are sure the paper has errors, publish your own analysis.

But in the end, mathematics can be just hard.

I'll note that some papers have errors that haven't been noticed after fifty years or more. No one really doubts the result, though a thorough analysis might prove them wrong. Automated theorem provers/checkers can catch some of that, but not all. But the human mind has limitations in how much detail it can manage and some proofs go beyond the natural limits.

| improve this answer | |
  • 3
    Can I ask you what exactly you suggest? To publish a correct solution to a long standing problem? Or to publish an "article" the whole purpose is to reveal a gap in the published paper? That seems more like a function of the designated review in the designated database. Hence my question. – mathreader May 23 at 20:35
  • Either is fine. But the reviewer can only comment on what they recognize. As I said, reviewers aren't perfect, though most are conscientious, especially for important works.. – Buffy May 23 at 20:36
  • 1
    Your answer and comment do not address one important aspect. As you noticed, math is hard. A person may spend half of their career trying to prove a small partial case of this big open problem. And their dedication for this project is severely undermined by the existence of a "proof" that is most likely erroneous, but is "validated" by the reviewer who most probably did a disservice to the community. To rephrase my question plainly: is there a way to make the reviewer do his or her job right and amend the published review? – mathreader May 23 at 21:25
  • 2
    +1 The reviewer's job is not to be perfect; the reviewer's job is to write a good-faith review. (Same for referees, editors, readers, and ultimately even authors.) Not noticing a subtle error is not a moral failing. The math literature is full of mistakes, large and small, and this makes life as a mathematician difficult, but given that mathematics is done by human beings, how else could it be?. – JeffE May 23 at 22:13
  • 4
    The MathSciNet reviewer is NOT an additional referee. – ZeroTheHero May 23 at 23:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.