Should I withdraw a paper if the Editor-in-Chief doesn't like it, but the reviewers think it's great?

Question

This is a long story. I am a mathematician. A couple of years ago I submitted my paper to one of the best journals in my field. After 8 months of waiting, I finally recieved a response, and it was "A major revision is needed". I looked at the reviewers' comments. They were very positive. The reviewers only pointed out some typos and suggested several little changes to improve the exposition. I thought that it would take me at most a day to revise my paper.

After that, I read the the comments of the editor handling my paper (let's call him Editor D for future references). The editor wrote:

The theory ... developed by the author has been well recognized by the two reviewers. Indeed it is easy to verify that the paper is well written and organized, and that, although not all results are new, there is enough new material worth of publication.

Then I found some additional comments of the Editor-in-Chief in a separate file. These comments shocked me. Here I must say that there exist several approaches to the problem (Problem P) that I studied in my paper. The Editor-in-Chief is the author of one of those approaches (Theory X). In his comments, the Editor-in-Chief wrote that any paper in which Problem 'P' is studied must be based on Theory X, and that I must completely rewrite my paper using Theory X. Furthermore, he wrote that my paper must address a much more general problem that includes Problem P as a simple particular case.

I felt that the comments of the Editor-in-Chief were unreasonable, to say the least. Therefore I withdrew my paper using "Withdraw" button at the website of the journal, and submitted it to another journal, where I, again, recieved two excellent reviews, and the paper was published shortly.

Recently, I came up with an idea about how to solve the more general problem, pointed out to me by the Editor-in-Chief in a very simple manner. I wrote a paper about it, which I am very proud of. In this paper, apart from solving the problem, I also pointed out, as clearly as I could, that Theory X, in fact, cannot be used to solve this problem due to some of its limitations. I decided to submit my new paper to the same journal, since it is one of the best journals in my field, and I had already published (or submitted) papers to other top journal this year. I admit that it was very stupid of me to submit the paper on a similar subject to the same journal. But what's done is done.

After several months of waiting, today I recieved a response, which once again was "A major revision is needed". Once again, the reviewers liked my paper very much. However, this time Editor D wrote in his comments:

By the way, a paper by the same author was processed by me as Associate Editor, with the requirement of a major revision; and then withdrawn and published in a different journal. I would not replicate this experience.

The Editor-in-Chief this time wrote a proper review of my paper which, in fact, was longer than what the reviewers wrote! In his review, apart from defending Theory X, and writing that I am completely wrong about its limitations, he also said that in his opinion I mix too many things up, and I must delete a half of my paper and rewrite the other half.

So, the question is what should I do given the "warning" from Editor D? I feel that deleting a half of my paper would significantly lessen its value. I want to withdraw my paper, but some people write that its unethical to withdraw a paper after peer review, and that the journal (and the publisher, which is very big) can put me into the black list of authors.

Edit-1: I would like to thank everybody for the interest in my problem and for the thoughtful comments. I've decided to resubmit my paper without any significant changes, and reply to Editor-in-Chief's comments explaining why I disagree with them. If the paper is rejected, then I will be free to submit it to another journal. If the Editor-in-Chief insists on a major revision, then I will withdraw my paper without a second thought and inform Editor D about the conflict of interests and the reasons behind my decision via email.

Answer 1

The basic issue is this:

The Editor-in-Chief's excessive involvement in your paper represents a serious conflict of interest on the EIC's part.

The reason it's a conflict of interest is that the editor-in-chief is a proponent of theory X, and wants to force you to rewrite your paper in a way consistent with his "pet" theory. This is entirely inappropriate behavior, because if the paper was written on a different topic, the editor-in-chief likely would not have intervened in such a manner.

Given that essential fact, it is entirely appropriate for you to withdraw your paper and submit elsewhere. At the same time, I would also bring this matter to the attention of the journal's publisher, to alert them of an editor-in-chief who is clearly overstepping bounds by inserting themselves into a publication in a highly inappropriate way. (I interpret the editor D's comment to be a warning to the editor-in-chief that he's not happy with the way things are proceeding. Therefore, I would explain to editor D in an email why you feel forced to withdraw the paper again.)

Answer 2

Your question splits into several, which I will try to address separately.

• Is it ethical to withdraw a paper after peer review? It depends. Sometimes authors use peer review to improve a paper, withdraw and resubmit to a better journal. Effectively, they exploit reviewers of one journal to improve their paper for another - clearly an unethical approach. Sometimes authors merely disagree with editors' decisions and do not want to make changes as suggested. In this case withdrawal is not unethical, but as Alexander Woo mentioned it would be nicer if you sent some final version to the editor asking them to accept or reject, instead of withdrawing.
• Is Editor D right to be concerned? Yes. Editor's role is to safeguard journal's resources (peers included) and making sure they are used for the best of this journal (and only then for the best of the whole community). As academics we may care about wider benefits first, but editor's role implies some administrative duties, and Editor D is right to be concerned about it.
• Is Editor-in-Chief right to promote his theory? No, I don't think so. First of all, I never heard of a Editor-in-Chief be so involved with a particular paper. In my experience, this role assumes leadership, not hands-on-work, so receiving comments from Chief is highly unusual. It is not clear it the comments should be treated simply as comments of another peer reviewer, or if they are more authoritative. This makes the whole situation a bit weird and possibly implies some potential conflict of interests here.

tl;dr: What to do? I would suggest to publish anything related to this line of research in other journals to avoid deepening the conflict of interest which is already sparkling between you and Editor-in-Chief mentioned.