I am currently reviewing an interdisciplinary computer science paper which is heavily based on previous works in logistics and process engineering that have been unfortunately published only in German (18 out of total 30 references). The overall idea is sound, but as the logistics structures are in some sense a basis of the paper, I have a strong inclination to reject the manuscript as the research cannot be followed and reconstructed by a researcher that does not understand German.

On top of that, virtually none of them are available online, or otherwise easily accessible.

I was going to hit the "Reject" button just a few moments ago, but I still have my doubts: I know that in humanities and also in mathematics people often cite publications written in other languages, but I have not yet seen something like that in computer science. Also, if it were one or two references, but 60%? The journal reviewing policy does not help me in this case.

I was thinking about proposing the following workarounds:

  • Extending the paper - this is probably not an option as the journal imposes quite a strong page limit, or
  • A supplementary report - the vital parts of the non-English text could be made available as an Technical Report or an on-line publication (I was thinking about arXiv) and this can be referenced in the manuscript.

Is it fair to reject it? Has anyone experienced something similar? What was your decision?

  • Comments are not for extended discussion; this conversation has been moved to chat.
    – eykanal
    Commented Oct 9, 2017 at 21:27
  • 1
    Could you expand on what you mean by "virtually none of them are available online, or otherwise easily accessible"?
    – 1006a
    Commented Oct 11, 2017 at 5:29
  • 1
    As the reviewer, are you fluent in German yourself?
    – arboviral
    Commented Oct 13, 2017 at 10:30

12 Answers 12


Having many non-English references is absolutely no valid reason to reject a computer science paper.

If a reader wants to follow up on the references, they can learn some German, find a translator, etc. But this is the responsibility of the reader, not the author.

  • Comments are not for extended discussion; this conversation has been moved to chat.
    – ff524
    Commented Oct 8, 2017 at 0:23

I tend to agree with @Arno's answer that citing non-English sources is fine. But that is actually beside the point, since as far as OP's situation is concerned, I think he has much less of a dilemma than he thinks.

The reason is that whether to publish a paper that references many German-language sources should be an editorial decision. As a reviewer, OP should check that the paper is correct (including looking up the references to the extent that's necessary, which OP can do since he speaks German), and that it is novel and lives up to the standards of the journal. If those conditions hold, he should recommend acceptance. He can also point out in his report the potential issue with non-German readers not being able to fully understand the theoretical background the paper makes use of. It would then be up to the editor to decide how to handle the situation, depending on their philosophy and the policies of the journal (which may vary, for example a journal published in Germany might have a different view of such issues than one published in the United States).

To summarize, the question of whether papers like this should be published is a very interesting one, but from a practical point of view, I don't think OP really needs to concern himself with it.

  • 3
    I am not sure that it is only about non-english sources. I have cites tons of non-english results, but it is easy to get hands to the originals or some translations. In the case of OP we are talking about some obscure and impossible to find on-line results. Also, if we expect the readers to learn new languages to understand results, why we have preliminaries and related work where we describe on a high level the techniques/results we need? If I am criticized that I do not explain what Gram-Schmidt Orthonormalization is (yes! it happened), why not the authors of this paper?
    – PsySp
    Commented Oct 6, 2017 at 17:59
  • 7
    @PsySp sure, there are other considerations besides the language (although whether something is available online is in my opinion not a valid consideration; plenty of sources, textbooks in particular, are generally not available online, and no one thinks that this is a problem). But as I said all of those are issues for the editor to consider, and I don't think they should influence the reviewer's recommendation, as long as the reviewer is able to verify the correctness of the paper and evaluate its importance.
    – Dan Romik
    Commented Oct 6, 2017 at 18:28
  • I agree with you that the reviewer can make suggestion and explain the issues and then is up to the editor to decide what to do. In any case everybody has focused on the "german/foreign language" issue which to me is not the real issue here. For me is pretty clear: if we cannot even find and/or reproduce the results, the paper looses a lot of credibility and in any case it doesn't help the authors increase the visibility of their result.
    – PsySp
    Commented Oct 6, 2017 at 18:40
  • 60
    obscure and impossible to find on-line results — There’s this brick building down the street with a bunch of books in it, where you can find people who are actually paid to track down copies of “obscure” references like the ones you describe. I believe they’re called librarians.
    – JeffE
    Commented Oct 7, 2017 at 14:17

Your only responsibility as a reviewer is to check that the article is scientific, relevant and original enough. When references are valid, the language should not matter, especially if - as you say - you can check them because you personally do know German.

In other words: I too am shocked that this is even an issue.

  • 10
    Your shock aside, I think it's good that such a question has been asked and so unanimously answered, since it's likely many reviewers out there harbour such misconceptions.
    – beldaz
    Commented Oct 11, 2017 at 19:36

See it the other way round. Many German papers have English references, although not all readers will be fluent in English. If you cite from a foreign book, eg. chinese it is helpful for the reader to give a (rough) translation.

If you reference to a paper from an 'exotic' language to translate the tile and give a very short synopsis of the content. Because it is helpful to know whether the reference is a paper on a specific problem or giving an overview over another topic.

If you are restricted in length in a journal, create a small website, were you provide the German original sources and make it readable to the english reader, by letting out the German citations.

Is the paper still valid with many foreign references? As a informed reader I would ask whether the reference given are available in the language of the paper. E.g. referencing to a standard algorithm in a german book, when the same algorithm is described in many english books. If this is the fact I would be more alert concerning the validity of the paper.

However brilliant papers have been written in China, behind the iron curtain and so on, without any reference to english sources.

I'd be happy to read a such brilliant work


From a scientific/scholarly point of view, there is absolutely nothing wrong with non-English references which date from a time when English wasn't quite as dominant yet, or which come from a field where it still isn't.

More recent non-English references can be a red flag but they are still no reason to reject a paper. They are a red flag because it's just not normal any more, and so people who publish in a language other than English anyway are likely to be very inexperienced, very old, very eccentric, or trying to hide plagiarism from English sources (possibly self-plagiarism). All of these factors are reasons to have a close look, which requires being able to read the references. To some extent these factors apply even when the references are from a technical field that hasn't standardised on English yet, as this makes it likely that the field itself has low scientific standards.

In any case, however, if for some reason you cannot really review a paper, then you should refuse doing so. Your inability to read some key references would certainly be one reason to do so. Or you could delegate the review (or just checking the references) to someone who can read the key references that you can't.

Although this was historically not done, I guess nowadays one should be very careful when citing non-English sources. Ideally, the author should be very explicit about the major claims taken from such a source. This way most readers can simply take it on faith that the source actually supports the claim, and those readers who can read the source (hopefully including a reviewer!) will know if it doesn't, without having to fully understand how it is used.

  • 15
    "Your inability to read some key references would certainly be one reason to do so" No, it certainly would not. Not unless they're written in sign language, and even THEN. I'm shocked that people here bow to Mighty English. Then there's the whole "can't even find it online". Huh? PAY up or go to the library. Commented Oct 7, 2017 at 2:46
  • 12
    @GwenKillerby Did you actually read what was written? It was given as a reason to not review the paper, since clearly not being able to follow the references makes one unqualified to do so. Also, it should never be necessary to pay to access references as part of reviewing. Commented Oct 9, 2017 at 5:01
  • 2
    @TobiasKildetoft If you shouldn't have to pay to access citations, directly or indirectly, then nobody could cite Science, Nature, Cell, PNAS, Physical Review Letters etc.
    – user71659
    Commented Oct 11, 2017 at 21:10
  • 3
    @user71659 You seem to be mixing up some things here, and also don't seem to realize that most researchers do not in fact need to pay to access any of those. I specifically said that you should not need to as part of reviewing. Commented Oct 11, 2017 at 21:18
  • 6
    @user71659: The reviewers are doing a favor to the journal, they shouldn't have to additionally fork over money to access paywalled references. Rather, the journal (which is being paid handsomely either by authors or subscribers) should foot the bill. Tobias is not saying that no one incurs costs accessing the references, we're talking only about the reviewers.
    – Ben Voigt
    Commented Oct 12, 2017 at 4:42

I'm not sure how computer science publications work, but having used quite a few non-English papers myself I have the following suggestions.

Almost every paper adds a small increment to the specific subject. Usually, authors can easily find one or several English language papers connected to the non-English paper and cite those instead to support their arguments.

In case the authors cite data from a non-English article, it quite acceptable. In this case they must reference strictly numbers that can be looked up in the article by a reader who doesn't understand the language. If there is a need to cite methods or interpretation of the data, then they should be translated and briefly summarized in English.

In any case, the burden of translating and verifying foreign texts should not fall on the reader of a journal.

  • 8
    From a physics point of view, careful citation of foreign-language sources is fine. Citing data is one case. Another might be citing a major result (such as an algorithm) but in that case I prefer to see the citation of the original paper followed by a clear citation to an English source that discusses it. That of course isn't always possible
    – Chris H
    Commented Oct 6, 2017 at 10:46
  • 12
    -1 for suggesting avoidance of citation of non-English papers because they're not in Engish.
    – einpoklum
    Commented Oct 8, 2017 at 8:13
  • 4
    Have I understood your answer correctly as saying this: "When an paper written in English wishes to cite a result first published in a different language, it would be best to cite a paper written in English that explains the result"? I'm pretty sure such a paper must, by necessity, cite the original source, which puts it right back into the category of "paper written in English citing a foreign language source". Is the paper being reviewed the chicken or the egg?
    – Ben Voigt
    Commented Oct 10, 2017 at 18:27
  • @BenVoigt There's no contradiction in saying that an English-language source should be cited if one is available. Obviously, the first English-language paper to refer to a particular result fails the "if one is available" test, so there is no paradox here. (I disagree with the answer, but your claim that it leads to paradox is incorrect.) Commented Oct 13, 2017 at 15:25
  • @DavidRicherby: You misunderstood me -- I'm saying that unavailability of an English-language source is not as rare an event as the answer suggests, as it must happen at least once for each foreign language paper. (Alternatively, "if the set of English papers citing a result first published in another language is non-empty, at least one of these papers is reviewed before any of them are published") Sure you can use circular citation to ensure all papers written in English cite another paper written in English using the same result, but that doesn't help the reviewers.
    – Ben Voigt
    Commented Oct 13, 2017 at 16:26

I like your idea to propose the mandatory provision of a supplementary report.

At the cost of some extra effort for the authors, this would be a good compromise between the interests of the authors (getting a fair evaluation of their actual work) and those of the readers (being able to reconstruct and extend the research).

An included benefit for the authors would be the possibility to disseminate the underlying ideas to a wider audience.

  • 17
    I would not call that "some" extra effort. Commented Oct 6, 2017 at 14:49
  • 5
    @PsySp "Readers cannot invest time to learn new languages to understand a paper." That is the fault of dumbed-down modern education systems. When I was a kid (in the UK) learning two foreign languages (and sitting nationally standardized exams in them) before the age of 16 was considered perfectly normal for anyone likely to be educated to degree level. We also learned a third foreign language in a less formal way from age 16 until starting university. And I don't recall any of the kids complaining that "it was too hard" or "it wasn't fair".
    – alephzero
    Commented Oct 6, 2017 at 19:08
  • 4
    @alephzero PsySp talks about a situation where you must learn arbitrary new languages on the spot to understand a paper, while you talk about learning two or three specific languages in school. Since you grew up in the education system before it was dumbed down, you should have the logical tools to find the flaw in your argument. Commented Oct 6, 2017 at 19:16
  • 12
    no no, no, the responsibility is the READERS, not the authors. Rejecting science based on the INCOMPETENCE of the reader is always wrong. Commented Oct 7, 2017 at 2:49
  • 7
    @PsySp The references cannot be found online, which is not the same as cannot be found. Libraries are there for a reason. Commented Oct 8, 2017 at 11:25

Rejecting a paper because you do not understand the language doesn't sound like a good excuse and people will get frustrated.

On the other hand, since it is not some standard and well known/used publication that has been scrutinized and/or translated, and it cannot even be found online it is up to the authors to make every possible effort and convince the readership that their techniques and relevant results are sound.

I suggest to give them the chance to enhance and update their manuscript with all the necessary details in order to judge the soundness of the paper. Failing to do so would (and should and a fair warning) lead to a regretful rejection.

  • 7
    "Some references cannot even be found online" is an absolutely ridiculous reason for rejecting a paper or suggest the kind of radical changes you're proposing. A huge fraction of the scientific literature, covering the first 90% of the history of science, is unavailable online, but it can easily be obtained by getting up and going to a library, or mechanisms like inter-library loan that were how science did its business for much of its history (and which seem to have worked perfectly in OP's case). Does that mean that all of that archive is now off-limits to citations?
    – E.P.
    Commented Oct 7, 2017 at 21:42
  • 7
    You're proposing that the authors spend a significant amount of work, thereby taking time away from useful research, to make up for a perceived inability of the readers to use inter-library loans in the same way it's worked for decades, and I don't see how you can think that that's a minor change. It feels like you expect readers of scientific literature to have everything spoon-fed to them, from language to availability, but that's not how the research literature works.
    – E.P.
    Commented Oct 7, 2017 at 21:59
  • 2
    Reading the scientific literature has always entailed a certain amount of work. You're entitled to your opinions, but I will just note that if I got a journal rejection based on the kinds of opinions you've expressed on this thread, I would take it as a good reason to take my publications to some other journal that does understand how science and its literature is structured.
    – E.P.
    Commented Oct 7, 2017 at 22:05
  • 3
    That is the authors' concern, and possibly the journal's, if they really do have a consistently high standard of accessibility to maintain. It is definitely not the reviewer's role to reject papers on this basis.
    – E.P.
    Commented Oct 7, 2017 at 22:07
  • 3
    @1006a OP's first comment on the question confirms he was able to obtain offline all the references required to verify the paper. That is not what "unverifiable" means. Your position would imply that references that are that easily accessible, comprising a large fraction of the literature, are off-bounds to further use.
    – E.P.
    Commented Oct 8, 2017 at 7:43

I would not go for a reject for this reason.

As a reviewer, wishing that authors provide more background from foreign language papers is acceptable. I would do so in this case.

Like, in normal case it is straightforward to assume that the reader can inform herself on previous techniques from earlier papers and you present the state of the art more to highlight what you have done differently.

Here it is not like this. So, the authors might be required to present a better/broader overview of the related work and state of the art. As a reviewer I would basically ask them to retell the essence of the foreign-language papers, so their paper is understandable and makes sense even if the reader cannot read the foreign language references.

As a side note: where it mattered, translations were published. You paper authors might not be aware of them, but they might be out there. I know that some Russian math journals were routinely translated in English during the Cold war. It's another issue how to cite such works in an English paper...


I would recommend rejecting any paper that is incomprehensible to somebody in the field without following its references.

The references not being in English isn't the problem. Leaning on the references too much is.

  • I would add: leaning on references that you cannot even find online!!
    – PsySp
    Commented Oct 6, 2017 at 18:04
  • 12
    @PsySp Get thee to a library already. Complaining about references not being available online is pure laziness.
    – JeffE
    Commented Oct 9, 2017 at 13:45

What is the purpose of references? Certainly allowing the average reader to follow the research is a part of it, but the more important part is giving appropriate credit to where the authors got their ideas, which may well include papers in the authors’ native language or which happened to available to them but not the wider internet.

In itself, definitely not a reason to reject.

That said, the paper should stand reasonably well on its own - the text should include sufficient summary that only those particularly interested in the details need look for translated versions of the citations. Also, a qualified reviewer should be able to confirm that the cited paper says what the authors claim it does.


There is no simple answer to this question: either you are able to actually judge on the quality of the cited material, or you are not in a position to be the reviewer for the particular paper.

Citing inaccessible material is simply slightly negatively affecting your evaluation, that's it. You should definitely reject if the author cites a simple corollary of a theorem to be found in a privately circulated memoir of the Slovenian Philological Society, 1883. This citation cannot be serious in a CS paper unless the paper is doing research on citations or invalid proofs. On the contrary, citing Gauss, Leibniz, Euler, Grothendieck, Kantorovich, Fichtenholz, or Kolmogorov is actually not a bad sign by itself. In general, you never know before you look.

In the particular case, since you happen to be fluent in German, there is less excuse for not being able to produce a good review.

  • The OP stated that he speaks German and was able to understand one of the German references that he was able to track down. Commented Oct 11, 2017 at 13:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .