17

I am reviewing a paper from a fairly famous journal in computer science. I am mostly sure that the paper was generated by AI. The paper has many problems and deserves to be rejected on the merits. But it is difficult to prove that it was auto-generated: I couldn't find any appropriate free tool or website to check the whole paper, and this would be useless anyway if the text was paraphrased.

What should I do in this situation? Can I write my guess about this plagiarism and give reasons confidentially to the editor to decide himself? It seems like if I merely recommend rejection, the authors will not face any consequence for auto-generating nonsense and wasting everyone's time. But I am not sure whether it is ethical to mention my unproven (and likely unprovable) suspicions.

Details:

  • Many basic requirements of a scientific paper are missing: no hypothesis, only a basic mathematical foundation, no experimental results, and the organization is very strange. Even the citations seem unusual.

  • The authors have previous papers which show that they are totally familiar with the structure of a scientific paper and scientific language. However, one of their recent papers does seem to have many of the same problems as this paper.

Improve this question
2
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on Academia Meta, or in Academia Chat. Comments continuing discussion may be removed.
    – cag51 ♦
    Commented Aug 4, 2023 at 21:37
  • 1
    m123: I have edited to greatly shorten this post. If I got any of the details wrong, you may fix them. But please avoid reverting to the super-long version; it's important to distill the question down to something concise and readable. I also removed your side-question about whether AI can learn from SE posts; you can make a separate post about that if necessary.
    – cag51 ♦
    Commented Aug 4, 2023 at 21:40

6 Answers 6

Reset to default
42

You seem to have sufficient reasons to reject the paper on its (de)merits, whether or not it was written by a bot.

You can certainly tell the editor that you are suspicious.

I doubt that the editor has software (free or otherwise) that could tell whether the paper was bot generated.

I wonder if the many citations are real. Chatbots are notably nonsensical in that realm. You could quickly check a few. Even if real they may be irrelevant.

Finally, LLMs can indeed scrape SE posts. Whether they could use that information to learn how to do better is unclear.

Edit in answer to comments from the OP.

Just describe the mess you found when you tried to check the references. You do not need to prove anything.

It will not reflect badly on you if the editor disagrees with you (though it may reflect badly on the editor).

Improve this answer
12
  • 1
    Thanks for your guide. Since the citations are dense at one section, they can be added manually. Although, some of them that I checked does not contain the cited quotes, their titles are apparently related. Additionally, some of the cited papers does not contain the title of the paper. They only contain the name of the author, the journal name, pp, and year. But, there are also some cited papers that contain the cited paragraph. (The passage is not coherent at all and the paragraphs do not follow any specific order, goal, or coherency. They are some independent paragraphs).
    – m123
    Commented Jul 21, 2023 at 0:10
  • 41
    @m123: I feel like you are way overthinking this. Obviously it's a bad paper. Briefly explain why it's a bad paper, click the "recommend reject" button, and move on with your life. If you're going to devote this much time and attention to reviewing a paper, spend it on one that actually has a path to being published and benefiting the community in some way. This one does not.
    – Nate Eldredge
    Commented Jul 21, 2023 at 2:36
  • 3
    @Nate Eldredge: Yes, I am overthinking this. You know, I am thinking to the unethical part of the action of such authors. This time we can reject it because of its poor context. But what if it wasn't like that? should we ignore the immoral part and just judge the correctness of the results? without considering its authenticity? It seems that we can even publish the correct outputs of AI as long as useful to read, but even in this case I think it must contain the name of the used "AI" as the author, not others. If I share my suspicion, can it be useful to prevent such a thing in the future?
    – m123
    Commented Jul 21, 2023 at 9:29
  • 3
    @NateEldredge On one hand I agree that you shouldn't spend much time on it. On the other hand, if somebody in academia is just generating papers to get cheap publications, I think the community should know about it.
    – Kimball
    Commented Jul 21, 2023 at 14:14
  • 3
    I would leave this up to the editor. In this day and age, when they get a report that a paper is nonsense, they should always consider the possibility of it being machine generated. I don't think they need you to point it out. In any case, the editor would have to judge it for themselves before taking any action.
    – Nate Eldredge
    Commented Jul 21, 2023 at 16:24
9

Between Conversational AI afficionados, the questions about the value of GPT-generated text (i.e., about copyright, ownership, merit, ethicality etc.) are hotly discussed, so you may not find a single value answer on this.

My take currently is this: GPTs are a tool like any other. They are not persons or entities and thus cannot, for example, have rights regarding ownership or authorship. So if a human uses a GPT to create large amounts of text, that does not really matter in itself. What matters is whether the results are fit for the job or not.

There is no really good way to automatically determine whether a given text has been created by a GPT or not, at least since the likes of ChatGPT 3.5+. The default, regular, text, has been tried in blind tests, and humans were not easily capable to tell. And you can, using your prompts, strongly influence the kind of wording the GPT creates, so even if we had a tool to recognize the default language of a well-known GPT, the author could trivially flavour the text so that it would be impossible.

In this case, you found out that the paper is, basically, a pile of junk. This information, worded more professionally, is what the editor is interested in. You do not have to guess which tools the author used to come up with his drivel. It should not matter which text editor, which data analysis tools, which search engine, or which GPT they used. If using a GPT makes part of their process easier or quicker, then so be it.

Whether the usage of the GPT has to be disclosed is something which probably varies as well. If it is used just as a fancy evolved text editor (for example, you could use it as a "thesaurus" to just find different wordings for you; or to rewrite your content so it contains less grammatical errors or easier language), then a journal might be fine with it; if it somehow had a much higher impact on the actual content (frankly, hard to imagine at this point of time) then it probably would need to be disclosed. This would be up to the individual journal to regulate.

N.B.: there are of course areas where the usage of a GPT has to be tightly watched (e.g. questions surrounding copyright, or using personal or otherwise sensitive/protected data in the prompts and thus at least possibly disclosing them to 3rd parties), but that is not principally different from any other search engine and I'd say out of scope for this question. Copyright especially is an issue, in my estimation, that's more relevant to AI generated picture/movie material than text.

Regarding the question of whether us here talking about this could teach ChatGPT how to avoid this in the future... unlikely. It is always hard to predict the future, but that's just not how these GPTs work. ChatGPT's training is years old (and was so even when the hype started). Combining a GPTs massive training base with current text and events is the hot frontier of GPT development; but the more current info is usually handled in a very different manner than the base training, from a technical point of view.

If the developers/scientists of a GPT wanted to make their texts even harder to detect, they would not need our input here, but they would just work on their algorithms etc. directly (they know much more about how their software works in detail). They don't need to wait for the training of the next big GPT to somehow pick up on this StackEchange question. There is no intentionality within the statistic data that's fed into such a system, it's just thoughtless statistics on a scale incomprehensible to intuition.

Improve this answer
5
  • I do think that the style of a technical journal article is very unlike 99% of what large language models are trained on. And how ChatGPT and the like sound are pretty jarring in a formal journal setting.
    – Jon Custer
    Commented Jul 21, 2023 at 13:06
  • 2
    @AnoE: Yes, I think that even if we had such a tool, the authors could use it to change the passage in a way that it is undetectable by those tools. But about your 2nd paragraph, don't you think that at least it must be mentioned in the paper that "this result are generated by this tool"? The same as what reaserchers do when they use Matlab, or any other tools.
    – m123
    Commented Jul 21, 2023 at 16:57
  • Additionally, I suppose, and I have always heard that a scientific paper must show a scientific effort. Where is the effort when the text is totally generated by chatgpt? except for the efforts taken to change the appearance of the text to be undetectable at the first glance?
    – m123
    Commented Jul 21, 2023 at 17:07
  • @m123 The scenario that an LLM will output a scientific paper that passes any form of peer review in a fully automatic way (i.e. given a prompt like "Find an interesting way to extend the work by X and Y on Z") is currently not realistic, even in fields that do not require experimental work or where the experimental work can be done in code, and even if the LLM has access to tools (like CAS, theorem provers, code execution, web browsing). An LLM that contributes to a publishable paper interacts with a researcher and acts as part assistant, part writing aid, sounding board, or knowledge base.
    – Polytropos
    Commented Jul 22, 2023 at 11:43
  • @m123: I have added a paragraph about disclosing the use. Regarding "effort": suppose you are a total genius and a new proof that generations of mathematicians have worked on just comes to you in your sleep. You write it down with no effort, and send it to a journal. This would not lessen the benefit of the publication at all, right? I believe "effort" is to be seen here in a meaning that implies that the actual publication has some value in furthering the field; not necessarily that the researcher had a bad time writing it...
    – AnoE
    Commented Jul 24, 2023 at 8:24
5

Two options:

  1. You could write directly to the editor now, without finishing the review, to raise your suspicions.
  2. You could finish the review, and include your suspicions in the "confidential to editor" box.

The more convinced you are that the paper is AI-generated, the more attractive #1 becomes, since it saves you from having to finish the review. The drawback is if the editor says "continue to review anyway" then the author would also be waiting longer, so only do it if you're confident.

It's because you are alleging misconduct that your suspicions should be kept confidential to the editor. Don't send it directly to the author, it is not likely to end well.

I checked other papers of the authors also, they are experienced, and it seems that they should not be such unexperienced that doesn't know the most basic requirements of a scientific paper.

About this, there are a couple more tools that the editors should have access to that you don't: they can see which email the authors used to submit the manuscript, and they can check if that email matches the authors' institution webpages. The editors can also see the cover letter if there is one, and they might have communicated with the authors already for other things (e.g. on whether the manuscript contains any copyrighted material). They are better-positioned than you to investigate, so you should definitely leave it to them.

Improve this answer
2
  • Should I write some of the problems of the paper in the parts that the authors can see? Do you suggest 1 or 2: 1. I write to the boxes for authors my evidences with a ling list of its problems (e.g, the organization of the paper, lack of any real proposed scheme, non coherent passage,...) without explicitly mentioning that I deduced this is AI generated, and then just write to the editor that I guess this is AI generated". Or 2. I just briefly mention to authors that this paper doesn't have any contribution and experimental result. And then write all my reasoning confidentially to the editor?
    – m123
    Commented Jul 21, 2023 at 9:09
  • 1
    @m123 if you're doing my option #2, then I'd take an in-between of your two options: highlight the most major issues, add "I have a lot of minor comments, but I think the major comments are already sufficient to reject the paper", and leave everything else in the confidential-to-editor box.
    – Allure
    Commented Jul 21, 2023 at 9:56
4

I recommend reviewing the manuscript solely for suitability for publication in the journal you're refereeing for. If you have suspicion that the text was generated by AI, and you think the conclusion that observation would drive you to is different from the conclusion you reached based upon the suitability of the manuscript as an academic contribution, mention your concerns in whatever mechanism the journal provides for confidential comments to the editor.

Improve this answer
9
  • 1
    I did that. I reviewed it without assuming that it is generated by AI. But even there is no scheme proposed by them! I don't mean the scheme they have suggested is poor or trivial. There is no scheme at all!!! In the proposed scheme section, they have written some sub titles, and just wrote some basic definitions. For example, they have written:3. Proposed scheme 3.1. Function. A function is defined as a relation between a set of inputs having one output each. 3.2. Integer: .... 3.3. ...
    – m123
    Commented Jul 21, 2023 at 15:27
  • (It really has these titles!) This is the sole content of what is called "proposed scheme". There is no section for reporting experimental result, or any other kind of result. The paper event doesn't have such a title. no title, no content. no table, ....Just it is full of discrete definitions. (And the definitions are not new to the field. They are basic famous definitions)
    – m123
    Commented Jul 21, 2023 at 15:27
  • 2
    Given what you're saying, you should give the manuscript a very poor review that has nothing to do with any suspected use of AI (which is what I suggest in my answer).
    – Scott Seidman
    Commented Jul 21, 2023 at 15:31
  • 1
    FWIW, the tech has been commonly available for a handful of months, ethical viewpoints are still forming. There was a time when we though graphing calculators would kill math education, now math classes require them. If you feel compelled to let the editor know of your suspicions, even though the paper will likely be rejected on "normal" review practices, then you should do that. I would, again, do it in a "confidential to editor" manner. IMO, though, the authors will already have a negative outcome (a rejected paper) that is related in some way to their practices.
    – Scott Seidman
    Commented Jul 21, 2023 at 18:38
  • 1
    @m123: A function is defined as a relation between a set of inputs having one output each. --- And what exactly is a relation between a set of inputs? A set of ordered pairs of inputs might be marginally better, but then there's the problem that the output of a function is an input of the relation, or whatever . . . Unless someone carefully explains what a relation is and explicitly defines what it means for a relation to be a function, this "definition" is pretty much useless. Indeed, I would think more people know what a function is than what a relation is.
    – Dave L Renfro
    Commented Jul 21, 2023 at 18:39
2

I'd strongly echo the opinions of other comments and answers that you should just appraise the thing on its own merits (or lack thereof) rather than the smell of AI assistance.

It may be premature, or insufficiantly "humanist", to propose that a really good AI-generated thing should be considered good "despite" its being AI-generated... but/and, not so long ago, in the U.S., things done by women were not even seriously considered (in any sort of supposedly merit-based competition). And various well-known racist stuff.

After all, if corporations are legally entitled to being treated as legit political entities, why can't AI's be comparably respected? Own copyright? :)

I'm not really inviting debate on the larger issues, but, by-the-way, I certainly do not expect students to mention that they used calculators, though perhaps crediting specific software packages, so that other people can learn. Is that "giving credit"? I don't know.

Improve this answer
3
  • 1
    After all, if corporations are legally entitled to being treated as legit political entities, why can't AI's be comparably respected? Own copyright? - Well, in that case the AI should be listed as the author, so this would be plagiarism, right?
    – Kimball
    Commented Jul 21, 2023 at 23:22
  • @Kimball, exactly! Semi-seriously!?! Seriously?!? Maybe. I have no idea. (I hardly manage to cooperate with anyone at all, beyond my five-year-old iMac, with whom I'm on pretty good terms.)
    – paul garrett
    Commented Jul 22, 2023 at 1:57
  • @Kimball: And then, if AI did plagiarism, put it in a blacklist, or in a jail...
    – m123
    Commented Jul 22, 2023 at 9:37
0

Ai-assisted writing is a tool to write content.

Paper writing is a tool to share content.

The scope of peer review is to filter non-relevant content for the peers of research.

The paper you are reviewing has crappy content. Is the content even crappier because the authors are using wrongly a tool? their bad, not relevant for the outcome of the review. There are people that did much worse errors because they wrongly used a tool to defend their "content", see for example an infamous econ/finance paper where the idea was flawed, and its demonstration was based on flaw use of a famous aI (assisted Intelligence) tool.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .