I have finished my Phd in maths shortly. A few days ago, I have heard a revision decision from a very good journal where I submitted my paper. The comments from referees are generally fairly positive, and are not too difficult to address. However, I have found a mistake in a theorem proof, which the referees have not spotted. This mistake is difficult to fix, or may be unfixable. The theorem is the extension of the main result of the paper. I assume it is not an important result, as there are no comments or revisions from referees on this part. Then how to write to the referees to address this issue? I imagine it would annoy both the referees and the editor. When I am going to submit the revision, is it better to remove the wrong theorem or to keep it there? Many thanks.
You absolutely have to advise the editor and referees of the error, and either correct or remove the erroneous part. It doesn't matter how much it annoys them, or even if it leads to the rejection of your paper. Keeping it there would be totally unacceptable and unethical.
Just tell the editor straight out:
"I discovered an error in the proof of Theorem 1.2.3. It does not seem to be fixable, so this theorem should be removed from the paper. The rest of the paper is unaffected [assuming that it actually is]."
The editor will probably ask the referees to re-review the paper with this information. It might change their decision, or it might not. But even if it gets rejected, that's something you just have to live with: fix the paper and submit it somewhere else.
You seem to be assuming that the referees noticed the error but decided not to say anything about it in their review. I doubt that. It's far more likely that they didn't notice it at all. Referees are humans and not omniscient gods, and just because they overlook an error does not mean it is okay to publish it.
Referees are there to check quality of paper, contents of the paper, procedures, theorems and lot of stuff so they often miss here and there being humans.
Letting the error go to be published can have two outcomes.
1st in academia there is trust on authors, other refering your paper assume the theorem to be fine and that can lead other papers to have errors.
2nd some one can spot the error later on and can cite your paper im his work showing the error which can be a matter of shame for you.
So better inform editor that it is the case and removing this portion will have no effect or these these effects. Let them decide then.
You should notify the editor, who will notify the referee. Much better to discover the error now, rather than after the paper is published, with your name on it. Then you'd have to publish a retraction, or worse, have someone point it out to you (and others) in their own paper.
Since it is on an "extension," it will probably be no big deal. Most editors would rather have a correct paper with one less extension than a paper with an error. The referee will probably be relieved as well; s/he was supposed to spot the error and didn't. It's barely possible (not likely) that the referee can suggest a way to fix it.
People make honest errors from time to time. But when one is discovered and not reported, it is no longer an "honest" error.