124

I am handling a paper as an associate editor that proposed an algorithm that I find to be weak. In fact, I was able to show that a very simple, brute-force approach actually has a better running time than their algorithm. Therefore, I will recommend rejecting this paper. Do I have an obligation to share my proof that the brute-force running time is better? I want the higher-level editors to have confidence in the rejection, but it also occurs to me that I might be able to improve my own result and publish it independently. Is this a violation of ethics?

UPDATE: Well this certainly took off! I would like to add the following:

  • The overwhelming consensus is that it would be unethical for me to "scoop" the other authors, so I will not do that. The advice from all is greatly appreciated.

  • The journal is the top in its field, so we have to be extremely selective. The problem proposed is fairly interesting, but overall the paper does not meet our threshold.

  • When I say that brute force is better than their method, I mean it in a provable, big O sense.

  • The authors' proposed scheme is not only inefficient, it is written in a very confusing way. In fact, I asked them to compare their approach to brute-force as a way of helping them clarify their argument, and they did a bad job of it, which is what led me to look into it in the first place.

  • The fact that brute force performs better than their scheme is not totally trivial because it relies on a combinatorial argument that is not amazing, but not completely obvious either.

  • I will share my proof with the Editor in chief, but I have decided not to give it to the authors; I will consider publishing independently in the future if their work ever appears elsewhere.

  • Their paper is not on arXiv or any other website.

12 Answers 12

218

I think you are treading on thin ice, ethically speaking. Obviously you, as an editor, have no obligation to help the authors in any specific way, and you are free to tell them about your improvement or not, but rejecting their paper, taking the idea/problem, applying a different method to its resolution, and then publishing this under your own name seems problematic.

As an editor, you are usually expected to treat the papers you are handling as privileged information, and you are specifically expected to not use your knowledge of rejected papers to scoop the authors - which is exactly what you plan to do. Say you reject the paper with comments along the line "important problem, solution is too simplistic". The authors now go back to the drawing board, come up with a solution similar to what you had in mind, and get their paper accepted. If you publish your idea first they obviously can't do this anymore - you have effectively made use of knowledge you learned as an editor to pull the rug from under the people who initially thought of the research project, even if they did not do a great job with the first submission.

I understand that it sucks that if you told them about your idea you would be giving away information that, in different circumstances, may be sufficient for co-authorship. However, I would argue, as a reviewer / editor we sign up for helping the authors "for free" to some extent (that is, without expecting recognition).

If you really don't want to tell the authors about your idea your best hope is that the authors get their work accepted somewhere else. In that case you are free to write your follow-up paper and cite the original paper.

  • 17
    What about telling the authors, and subsequently collaborating on a paper? As it’s the editor’s idea, I don’t think they can reasonably be opposed. – JNS Nov 27 '17 at 14:18
  • 122
    @JNS "As it’s the editor’s idea, I don’t think they can reasonably be opposed." And that's also the problem. I would consider it majorly inappropriate if a person with so direct power over my manuscript would come with a proposal that, in essence, boils down to "I'll reject the paper, but if I can join in as a co-author we can talk about it again." – xLeitix Nov 27 '17 at 14:33
  • 14
    What about: “I must reject the paper given there’s a better idea I know about. I can share this and we can work on a paper for a separate journal where I have no influence?” – JNS Nov 27 '17 at 14:37
  • 72
    @JNS I don't think there is a variation of "I must reject this, let's work together on it" in which you don't come off as using your position to bully into a research project. – xLeitix Nov 27 '17 at 14:43
  • 19
    @KonradRudolph I would consider the ethical problem of not publishing a minor finding to be very small to non-existent. If we go down that rabbit hole, we quickly end up in a world where we would be acting unethically whenever we have an idea but don't find the time to work on it. – xLeitix Nov 27 '17 at 14:53
54

If the paper is overall lousy then simply reject it. I'm sure the reviewers would give you plenty of reasons for this.

However, if all it is, is a weak algorithm but otherwise well written, then it might still be worthy of publication (depends on the journal). Once published, you can then publish your own work and cite the paper. After all, your algorithm is indeed inspired by theirs.

Also, how sure are you that your algorithm is "better"? Always better? 90% of the time better? On CPUs? What about GPUs? How does it scale?

As an example, think about all the sorting algorithms out there. There is not a best algorithm. Depending on the dataset being sorted, any number of algorithms could give the best result.

17

Does running time actually matter?

It's a program for a paper, not something to be distributed and run commercially on many machines. Perhaps they coded it the way they did because that algorithm is more readable and understandable than a brute force approach.

For example, if something in my paper required me to do something simple like add up the numbers from 1 to n, we all know the sum is equal to n(n+1)/2, but having a loop that goes from 1 to n and adds them together is simpler to read.

Assuming this is the only 'lousy' thing in the paper that you take issue with, I don't see it as a reason to reject. If you could come up with a brute force approach that quickly, surely they could as well. Perhaps ask them why they chose that algorithm, or ask a senior editor at your paper for a second opinion, not random strangers on the internet.

As far as using the algorithm you wrote yourself goes, the other answers have adequately stated that that would be incredibly unethical.

  • 7
    "Does running time actually matter?" Usually to at least some extent. For example, I've published papers establishing that there's a polynomial-time algorithm for some problem, without pinning down the degree of the polynomial. But "polynomial" made it a significantly stronger paper than "exponential" or "no analysis". In the case of the paper in the question, the running time is worse than what is normally the dumbest possible algorithm. Being out-performed by brute force is usually really bad. – David Richerby Nov 28 '17 at 14:55
  • 7
    At the least, I'd expect the paper to acknowledge that the brute force solution is faster and to justify their alternative approach -- that is, if the authors were aware of this fact. If the algorithm is the major contribution of the paper and this isn't discussed, I'd withhold any recommendation of acceptance until the authors have clarified the manuscript. – jme Nov 28 '17 at 16:42
12

This is one of the most feared and contentious acts in the publication of science. We all have heard stories of the paper that was rejected, only to serve as impetus for a subsequent publication by one of the rejecting reviewers. The worst case, which you would not be guilty of, is rejecting the paper for the sole purpose of benefiting from the idea. That's not the case here. If the optimality of the algorithm is the point of the paper, and you can clearly demonstrate their logic is flawed, you have every right to reject the paper on those grounds. But you should lay this out for them in the review, and give them a chance to rebut. If your logic is iron-clad, you'll win the argument.

Whether it is ethically sound to then move forward with your own publication on the topic is unclear. Did the authors present this work at a conference? If so, and had you seen that talk, you would be justified in moving forward. If not, the manuscript is considered confidential, and your idea was spurred only through a confidential review process. It would be wise to discuss this with the general editor of the publication for guidance. Some day, they may review your own paper.

  • 4
    If the work has been presented in public, it's public. It's not necessary for the asker to have seen the talk: they might have been told about it by somebody who was there, for example. – David Richerby Nov 30 '17 at 13:38
3

I believe rejecting a paper just on the basis of poor performance of their algorithm and then publishing your brute force algorithm is highly unethical because: to me it sounds like you tried the idea of making an algorithm only after getting an idea from that work. Secondly, brute force algorithm are not science and they can never win over a smart procedure, it is not ethical to reject a research work to just get yourself a publication.

Then again, you can very well communicate to the authors that you have something better. Collaborate with them. Tell high level editors you can't handle this with as you are working together with them. That's ethical way to do it.

  • 2
    'Secondly, brute force algorithm are not science and they can never win over a smart procedure" - I think this is a sweeping generalization, and dismisses the opinion of the OP and the comments by e.g. David Richeby made elsewhere – Yemon Choi Dec 2 '17 at 15:14
1

Why not share the outline of your proposal in the review correspondence and ask the editor to share your contact details with the authors? In this way you entrust them with the decision to contact you and thereby solve the moral issue of bullying the authors (you re-establish the level playing field by letting them contact you).

In this way, they are the one who have to up the ethical stakes by responding in kind. They might not do it, sure, but your review letter will be known to the editor and the other reviewer(s) and this should nudge the authors on the safe side.

In any case, if the alleged existence of a brute force solution informs your opinion of the paper, you're duty bound to put that (and sketch some evidence of your claim) in your review.

Certainly, many reviewers have shared useful suggestion with me in the past. These can make for lovely diner table conversations in conferences later on should you chance on the person.

On the whole, you will not be worst off for it: it's not like you planned on inventing that brute force algorithm before you got privy information trusted to you.

I know it's a low bar, but it seem to me this course of action would at least pass the golden rule test (which I suppose is really why you are asking on a public forum about it).

  • 1
    The first two paragraphs sound like "I'm not bullying, just nudging from an advantaged position". It isn't clear to me why this is less of a problem. – trichoplax Dec 2 '17 at 10:35
  • @trichoplax: they are nudged. But no longer from an advantaged position (they are free to contact the reviewer and at that point the referee report/decision has been send to them. So back to being colleagues. No more from an advantaged position). – user189035 Dec 2 '17 at 10:53
1

Assuming that you do not especially care about the topic of the paper that you reviewed, and that the idea that you had while reviewing is not exceptional and did not require huge amounts of work, I think the best course of action is simply to write up concisely that idea as part of your review, and give it away to the authors of the original submission.

If they use it, I agree it's too bad that you won't have credit for it, but you will have gotten other people to polish and write up your idea -- which is probably significantly more effort than what it will take you to concisely describe the idea in the review.

I'd say the main problem is if they don't use it, and you'd like the idea not to die off. I don't know what's a good solution in this case. Wait for their paper to be accepted somewhere, or wait for several years to be sure that they have given up?

1

The first thing you should do is to "recuse" yourself from the editorial process. That is, have someone else in the organization make the accept/reject decision since your own work now creates a conflict of interest. Put another way, you have an "axe to grind."

After the decision is made, then you can make your move. Let's say that your paper makes a decision to accept/publish. Then you can publish your improvement later, citing the original paper.

If your paper's decision is to reject, then you can approach the authors with your improvement, and honestly tell them that you had nothing to do with the rejection. (That's why you need to "recuse" yourself.) If they agree, you might offer to "run interference" for a second try at your shop; otherwise, you and they might publish elsewhere.

0

You have a valid reason to reject their paper, but no obligation to share your information. If you wanted to pursue it as research, wouldn't the best option be to privately contact them outside of your official duties, and ask to work with them?

  • 1
    There are additional ethical complications to this recommendation discussed in other answers. – sondra.kinsey Dec 2 '17 at 17:17
-1

I think you have an obligation to academia to disprove his algorhithm with a brute force algorithm. Give your brute force proof to everybody and put it in the public domain. After you disprove it, if you have an algorithm that's better than his and better than brute force, you could offer to help him write a better algorithm. You and him can both be coauthors of the new paper with your new algorithm. Wait until everybody accepts that his original algorithm has been rejected before offering to coauthor with him the new algorithm. Tell him that you have a new, better algorithm, but don't tell him what it is until he agrees to coauthor with you in a written, signed document. That way, you get credit for helping, but are not taking away his idea. However, remember that some ideas are proposed in odd ways because of a need to avoid similar patents. An inefficient algorithm that is patentable is better than an efficient algorithm that looks like someone else's invention.

  • How do you "disprove" an algorithm with another algorithm? – John Bentin Nov 30 '17 at 12:42
  • @user10691 I guess Russell means something more like "disprove the usefulness". – David Richerby Nov 30 '17 at 13:38
-1

The fact that the article inspired your own results that you believe are publishable suggests that their article is also publishable. If their article is not good enough for the specific journal, you may recommend publishing it in a less prestigious journal.

-8

As addition to xLeitix very thoughtful answer I would give the authors a period of grace (say half a year or so, depending on the typical time scales of your discipline).

If by then they did not publish an improved version matching the performance of your ideas from now, I would regard the confidentiality period to be over and the subject to be fair game for everyone including you.

If they published the poor version somewhere else you should cite them. Do not copy from their submitted previous manuscript. Conduct your own research and present it as such.

If you tell them now about a possible way to improve it's because you want them to improve and publish a better version. If they don't, you could still do it on your own then. But you would have to wait to find out.

The waiting period is the key here and without it you are indeed acting unethically. The waiting period must be really quite generous. However, it should not be infinitely long because otherwise as an editor (or reviewer) you would unfairly restrict yourself in possible research if all topics you ever got to see were inaccessible to you in the future.

  • 15
    Confidentiality periods are not something open to interpretation. – Mast Nov 27 '17 at 12:45
  • 2
    Six months would be a very fast turnaround. Even if the authors wrote their new version today and submitted it, it probably wouldn't be publicly available within six months unless they posted a preprint. – David Richerby Nov 27 '17 at 13:57
  • 22
    Downvoted. There is no statute of limitations on the confidentiality of submitted results. Unless/until the authors publish their contribution, you cannot use it in your own work. In particular, you cannot claim sole credit for anything described in the paper, but you cannot cite the authors without breaking confidentiality. While this eternal embargo may seem "unfair", that's the risk you take when you agree to be a referee or an editor (or a grant proposal reviewer, or a thesis committee member, or...). – JeffE Nov 27 '17 at 14:52
  • 4
    Writing the new paper would require attributing the problem to the original authors, which can't be done as @JeffE explains in his comment. The new paper can't be written without using some of the original paper. – David Richerby Nov 27 '17 at 15:57
  • 4
    @DavidRicherby Maybe the problem is already known. From the question this does not become clear. – Trilarion Nov 27 '17 at 16:03

protected by Community Dec 2 '17 at 9:01

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.