21

For non-native English speakers, is it ok to use ChatGPT as a translation assistant?

Example prompt:

As a non-native english speaker I am writing a scientific article, so you are my english language proof reading services assistant: Please correct the following paragraph for spelling, grammar and style. Please do not alter the technical content but focus on improving the language only.

If the answer is no, then how is using AI-assisted translation support different from the generally accepted (even encouraged!) academic writing proof reading services for non-native speakers(*)?

Would it be also ok to let the AI tool not only improve English writing attempts but let the AI translate whole sentences or paragraphs from the author's native language (say French) to English?

Of course I would very critically review the model's output. I take it as given that any scientist's passive understanding of English is good enough to judge the technical, subject-matter related correctness of a writing assistant's output (AI or human), otherwise the scientist would not be able to follow international scientific discourse in the first place.

(*) e.g. from Submitting Articles to ACM Journals:

Language Services ACM has partnered with International Science Editing (ISE) to provide language editing services to ACM authors. ISE offers a comprehensive range of services for authors including standard and premium English language editing, as well as illustration and translation services

Improve this question
14
  • 12
    Someone using a language model for a task it's actually suited for! My faith in humanity is <strike>restored</strike> slightly less tarnished! That said, 1) you might try to find a dedicated translation interface without the chatbot stuff, 2) depending on exactly what the model was trained on, it might not have adequetely sampled the areas of the language where papers in your field live, and 3) that prompt seems a bit cargo-culty. I'd be somewhat surprised if it adds anything over saying "Translate from French to English, in the style of a scientific paper" (it's a transformer, not a person).
    – Ray
    Commented Oct 13 at 18:48
  • 13
    As an aside: Chat-GPT is not well suited to translation, there are tools using similar technology that are much better at it. Google for specific translation tools.
    – Jack Aidley
    Commented Oct 13 at 20:55
  • 7
    Transformers (the “T” in “ChatGP𝐓”) were specifically invented for Google Translate (and, in fact, their use as generators (the “G” in “Chat𝐆PT”) is what is actually the result of fortuitous happenstance). So, yes, this is indeed a rare case of “AI” being used for what it was designed for. That being said, since Google Translate essentially is ChatGPT (not literally, but the same underlying tech), the question basically boils down to “is it OK to use Google Translate as a translation assistant”.
    – indi
    Commented Oct 13 at 23:03
  • 3
    @DavidTonhofer Looking at message boards for translators, I get the distinct impression that their job market has massively shifted towards post-editing of AI translations since roughly 2023. I don't think it's plausible that this is due to specialised translation AIs.
    – Polytropos
    Commented Oct 14 at 8:49
  • 2
    @MisterMiyagi Yes, indeed, the training and configuration of every AI is different, though, as Ray points out, the core is the same. I do agree with you though: It would indeed be silly to use ChatGPT for this purpose, in much the same way that it would be foolish to use a mallet to drive a nail. I mean… it would probably “work” for simpler tasks, but… seriously, they should just use the right tool.
    – indi
    Commented Oct 14 at 14:24

8 Answers 8

Reset to default
34

Your two questions are quite different, but I'd give the same answer to both. At some point in the process it would be acceptable, though probably not for the final version (less acceptable). But, using it as a tool rather than as a replacement for thought is probably ok.

It is, however, dangerous, and more dangerous when the non-native speaker is especially weak in the target language. There are lots of problems with an AI tool. The most vital issue is that it is nearly (maybe totally) impossible to know what it does or how it makes its decisions. It has no mind and no human judgement. It has no morals or ethics.

However, IMO, an AI tool can be used as long as the output is vetted by a moral, thinking, human. If you have the knowledge and judgement to verify that what it produces is valid then it seems fine (to me) to use as a tool. But if it is producing output that you aren't certain is correct then the danger increases. And for some scientific work, that is not acceptable.

In other words, oddly, if you desperately need to use it, then you should not. If it is just a minor suggester of how to phrase things and you will completely understand the output (in other words, if you don't really need it), then it is fine. Odd, that.

Improve this answer
7
  • 2
    As I mentioned, there already have been professional proof reading, language and translation services for a long time. And of course I would critically review their writing/translations/suggested edits as well. I would take it as given that any scientist's passive understanding of english is good enough to judge correctness of the services's output (AI or human), otherwise the scientist woould not be able to follow international scientific discourse in the first place.
    – zx-81
    Commented Oct 13 at 10:41
  • 13
    @zx-81 You shouldn't take that as a given. I have an example, from earlier this year, where a fairly strong English speaker didn't realise that ChatGPT changed the meaning of the text: If I let ChatGPT improve a post of mine linguistically, does that count as "AI generated content"?
    – wizzwizz4
    Commented Oct 13 at 20:06
  • 3
    @zx-81 The person with less than ideal English is that it leaves you very vulnerable to tricky meanings. A paper is written to communicate, not to deceive. But you can't trust an AI not to write something deceptive.
    – Loren Pechtel
    Commented Oct 14 at 1:16
  • 3
    This is another great example why such AI tools are really bad at getting answers in a topic you don't know about, but are really great as search tools, in cases where an answer is hard to get but easy to verify.
    – vsz
    Commented Oct 14 at 5:28
  • wizzwizz4: Thanks for the link to the relevant post. This is a very good example. (However, I for myself did catch the changed meaning of the AI output immediately).
    – zx-81
    Commented Oct 14 at 8:09
12

Academia is still figuring out how exactly to handle AI writing tools, and there are some uncharted issues here, eg how to handle it if ChatGPT is used so heavily that if a human had done it, they'd be a coauthor.

Edits for spelling, grammar and style are clearly unproblematic. If an academic text is translated by a human, the translator would receive credit, but not co-authorship. As such, I personally feel that if an AI-tool is used for translation, this should be mentioned in the article, but it wouldn't be an ethics issue.

What is important though is that the responsibility remains with the author. This means that the author needs to verify that the output from the AI model indeed says what it is supposed to say. The way how ChatGPT makes mistakes seems very different from how humans make mistakes, which can easily mislead us into trusting it more than merited.

For example, the pieces of mathematical writing by ChatGPT I've seen superficially read like text an actual mathematicians would write. There seems to be a logical structure there - until you try to follow it, when it becomes clear that it is actually all nonsense. In contrast, student-written nonsense tends to be glaringly obvious. Thus, when checking AI-written text for accuracy I believe we need to be much more sceptical than we would be when checking the work of a fellow human.

Improve this answer
2
  • 1
    I don't think your characterisation of LLM mathematical capabilities still holds up, see e.g. mathstodon.xyz/@tao/113132502735585408 .
    – Polytropos
    Commented Oct 14 at 8:42
  • 4
    @Polytropos My characterisation is explicitly about the stuff I've seen, not about the stuff that exists.
    – Arno
    Commented Oct 14 at 9:17
8

I only review papers in engineering & physics fields, and I suspect the answer depends on your particular research field.

When I, personally, am reviewing a scientific paper, I actually have no interest in, nor attempt to find out whether AI was used to generate any portion of the paper.

What I am reading for is to find out whether the process is scientific, logical, readable, and whether conclusions are proven by the data acquired, whether the data acquired makes sense and could likely be replicated, and that the experiment performed has its Purpose well-defined by prior research/work.

I personally try to be forgiving to non-native English speakers, I am more interested in their thoughts than their writing. What I really care about is whether the research will help other researchers. In engineering/physics, minor grammar mistakes may not cause any problems with understanding the experiments, and can sometimes be a question of style preference more than scientific communication. (If it makes the experiment hard to understand or has double meanings, I request the writer to clarify.) On the other hand, for sociology or philosophy, for example, the exact use of language may be a critical part of the actual research, so may be scrutinized much more closely.

So, as others have said, as long as you can proof read the LLM’s output with a critical eye, and ensure it is scientifically valid, then I personally see no problem with using ChatGPT/other LLM’s to help you proof read your paper. (In fact, I hope it increases the number of good scientific publications & scientists by lowering the language barrier!) For the current state of AI LLM’s like ChatGPT, you will need to be extremely careful that they don’t change your arguments or conclusions. Perhaps it would be worth buying the upgraded OpenAI/other services to increase their accuracy.

Improve this answer
5
  • " In engineering/physics, minor grammar mistakes may not cause any problems ", " more interested in their thoughts than their writing" Ye., of course. But in our extremely competitive publish-or-perish-world, the style & language (all other things equal !) may make the difference, for a paper to be accepted or not.
    – zx-81
    Commented Oct 14 at 8:10
  • Style and prose is not the point of a scientific/engineering paper, so personally as a reviewer for Optica/JLT/other photonics pubs I try to avoid having the author’s style affect the acceptance. Other reviewers might think differently on this point though.
    – Demis
    Commented Oct 15 at 15:37
  • 1
    @Demis avoiding to have style affect acceptance is reasonable as long as the writing is clear to understand. But some papers have so poor grammar that whole passages become needlessly ambiguous or at least take way longer to figure out than they should. It's not the job of peer reviewers to first spend that extra time and then help the authors polish up the text, nor does it do the authors any favours to leave the language problems in the published version.
    – leftaroundabout
    Commented Oct 15 at 16:41
  • Welp, as of 2024, on the one hand, the ignorance towards the actual authorship or predominant presence of an LLM due to lack of sufficient enough validation tools is understandable, yet on the other - the chance of letting an illiterate to the outer world is fairly tremendous... However, if we set full responsibility for LLM usage on a student, in the aftermath, it's their choice how to exist in the society, I believe. The what is actually incredibly sorrowful though, is alive professors reading, scrutinizing, and realizing output of a machine and not human...
    – Artfaith
    Commented Oct 16 at 14:43
  • I don't see how peer review would accept a paper with "whole passages becoming ...ambiguous" nor "illiterate to the outer world" - if that's occurring then the reviewers (and editors) are not doing their job, the journal is thus not terribly good with low standards, and LLM usage could only make a positive difference from this low starting point!
    – Demis
    Commented Oct 18 at 5:28
4

Translation should be a relatively uncontroversial use case, as at least in theory the translation engine just takes ideas that are already in the source text and represents them faithfully in the target language. The machine just acts as an intermediary and improves scientific communication if it can speak the target language better than the author. The author obviously remains fully responsible for the work, but they are also in the best possible position to check. Besides the danger of misrepresentation of the intended content (which I think the author can check very well) there is also probably some risk that the translation system picks a wording for things that have been said many times before that comes from another author (maybe from a widely used textbook on the subject), and I think this is significantly harder for the author to check/prevent; but this type of regurgitation should only affect boilerplate material and I could see it happen with a human translation assistant, too, especially since it is my understanding that professional translators use both AI and non-AI translation aids themselves.

That said, the rules for AI use in academic writing in general are still evolving, and may evolve differently in different disciplines and for different use cases; and they will continue evolving as model capabilities increase. It could make sense to check the guidelines of the journals you intend to publish in to be sure what rules apply currently to your situation.

Improve this answer
9
  • 7
    RE: "at least in theory": In theory, theory is the same as practice, but not in practice: various attributions. If you can't verify how it arrives at a "decision" the results cannot be trusted.
    – Buffy
    Commented Oct 13 at 12:10
  • 2
    What you describe is the ideal of AI-powered machine translation, perfect success and accuracy only limited by the abilities of different languages to express some concepts. If we had that, it would be vastly less controversial, especially for uses like translation. We don't. Especially with ChatGPT-style LLMs, big errors are not rare.
    – Peter Cordes
    Commented Oct 13 at 19:00
  • @PeterCordes: I explicitly said not to trust, but to check. For the record, I would strongly advise checking also if a translation/post-edit were done by a human with subject expertise. In terms of error rates, I've had to occasionally post-edit technical texts (computer science) translated from foreign languages to English, both by professional translators (in the pre-LLM era) and by GPT-4. I would say GPT-4 was a lot better in terms of post-editing effort required than unaided non-subject-expert human professional translators.
    – Polytropos
    Commented Oct 13 at 20:41
  • @Buffy I'd disagree with the claim that in theory, theory is the same as practice. In fact, I would say it is clear that in theory, practice and theory are very different, but the unreasonable power of mathematics shows that in practice, theory when done with care works surprisingly well! I would also disagree that black-box outputs can't be checked without turning the black box into a white box. There are many cases where checking a result is much easier than achieving it, and translation is one of those cases (although it is not the most striking one).
    – Polytropos
    Commented Oct 13 at 20:47
  • 3
    I simply said that if you don't know how an AI arrived at an answer, then you can't trust that answer. Verification needs to come from elsewhere. The same is true, actually, of an answer given by a human.
    – Buffy
    Commented Oct 13 at 20:50
2

One concern with using AI tools like ChatGPT for translation is that subtle changes in meaning may occur, and the author might not notice them. However, this issue can arise even when the author translates the text themselves, as non-native speakers may inadvertently introduce errors or nuances that differ from their intended meaning. In this sense, the risk isn't unique to AI, and I don’t see a major difference between using AI for translation and the author translating it manually.

With that concern addressed, I see no ethical or practical issues, provided that the author remains vigilant in reviewing the AI-generated text for accuracy. (Which one might be tempted to forgo if it seems to work most of the time...)

Improve this answer
1
  • 3
    I’m having a hard time buying those comparisons to manual or human translations. Doing the translation oneself means one actually writes the text one wants. Using a human translator means one has a very slow turnaround with a human being that can provide feedback on ambiguities. With an LLM it’s all done a matter of seconds, sometimes even directly integrated into tooling. The level of attention - vital to catch changes in meaning - is not at all comparable across these options.
    – MisterMiyagi
    Commented Oct 14 at 9:36
2

By now many publishers and conference have laid out their rules for the use of LLMs. Most of the ones I have read specifically allow LLMs and other automated tools to polish your text.

The following example is taken from AISTATS call for papers (emphasis mine)

LLMs and image DGMs are not allowed for the following use cases:

  • Fully automatically generating text of more than one page, unless the produced text is presented as a part of the paper’s experimental analysis.
  • Generating quantitative figures (such as learning curves), unless the produced images are presented as a part of the paper’s experimental analysis.

Other potential use cases of LLMs such as polishing text (e.g., paragraph-wise, prompted by a manually-written paragraph of content) are not banned.

Even with the usage of LLMs and DGMs, it is still the authors’ responsibility to ensure the quality, correctness, and originality of their submission(s).

The IEEE Robotics & Automation Society writes (emphasis mine)

We encourage the use of those emerging technologies in a responsible manner. We aim that such AI tools mostly promote researchers' own capacity to create high-quality scientific work. For instance, AI tools can help researchers arrive at new ideas and improve self-written texts, especially for non-native speakers of English. However, we need to consider that AI tools also raise questions about what exactly constitutes their responsible use.

So the answer depends on the specific scientific community and your target venue.

Improve this answer
1
  • 1
    Your conclusion is flawed. Note the term "responsible manner" in the IEEE statement. It depends on more than venue. Much more.
    – Buffy
    Commented Oct 14 at 13:10
1

Check out this story on the Science Friday podcast from 20 September and titled How Are AI Chatbots Changing Scientific Publishing? In it, a senior executive editor at The Lancet says that large-language models can be very helpful for non-native speakers of English. She even argues that this is important for equity.

Of course, any user must be scrupulous in avoiding any ethical issues from leaning on AI to such an extent that the resulting artifact is not accurate or not truly one’s own intellectual accomplishment.

Improve this answer
0

When someone tries to proofread a paragraph and the AI tool lacks a holistic understanding of the context, it may provide inaccurate suggestions that lead to undesired changes in meaning. The key here is to assess whether the proofread paragraph aligns with the broader context of your writing.

I believe the debate is ongoing about the extent to which AI tools should be used in academia. From my perspective, if you have a good grasp of the target language, using AI for stylistic changes to improve the flow of your writing is beneficial. However, the challenge arises when people start using AI tools to generate content rather than for linguistic refinement. This becomes problematic due to the speed at which AI creates information (whether accurate or not) and the slower process of peer review, for example.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .