9

So simple question for those that already reviewed for MDPI: how did you feel about the quality of the other reviewers (in general)? I have reviewed a handful of papers now as I feel it is necessary (in the end others review my papers too), but I am starting to have serious doubts about the "level" at MDPI. Maybe it depends on the journal/paper etc, but for most of the papers I reviewed it ended in me feeling rather idiotic to even bother. (on a side note: I avoid MDPI to publish in, it is really my last resort to publish something in, if I have no other option, but this does not really happen).

To summarize: One paper was a copy paste of another paper, they just changed the molecule they used. Now I can get this, but there is a difference between bluntly copy pasting (even certain mistakes from this previously published paper in MDPI) were kept(!!). I was the only reviewer to actually review this paper. The other 3 pretty much just gave it a go to be published. Which made no sense, the language was awful, there were critical errors in the paper. On top of that, after review round 1, all of a sudden there were extra authors on the paper (and the editor never said anything about this even though I mentioned this in my review report).

Another paper: 2 reviewers, I had some minor remarks, but still some issues and the paper also had several issues language wise. The second reviewer just accepted the paper as it was and congratulated the authors on the splendid work. The paper was mediocre. Nothing special and it contained some errors.

A third paper: I wrote a long list of errors, small things, some bigger ones. In general the paper was again mediocre but ok. It could be published if they removed these issues. The 2 other reviewers barely mentioned anything. They just had a few points (like 3-5) that were really minor things. It seemed they never really bothered to read the paper.

Now I know this happens in other journals as well. I even reveiced "reviews" from reviewers while it was obvious they hardly bothered to read my paper, but MDPI seems to top the cake to be honest.

The duality I am facing now is: should I bother to review for MDPI again (and at least try to improve the manuscripts) or not bother anymore and only review for other, more reputable, journals? I might be a bit naive as I know other (reputable) journals might also have shitty reviewers.

I specifically ask about the experience of reviewing for MDPI, not about their reputation MPDI in general.

10
  • 4
    Should I accept review requests from dubious journals? seems more relevant than the duplicate target used for the question's closure.
    – Anyon
    Commented Jan 5, 2023 at 13:20
  • 5
    Does this answer your question? Is MDPI a reputable Academic Publisher? Commented Jan 6, 2023 at 14:04
  • 4
    Reviewing experience is the best way to judge if MDPI is reputable, so this is about whether MDPI is reputable. Commented Jan 6, 2023 at 14:05
  • 1
    @Allure I disagree with the reopening, because OP asks "should I bother to review for MDPI again (and at least try to improve the manuscripts) or not bother anymore and only review for other, more reputable, journals?" which is very close to asking if MDPI is enough reputable since there are other journals more reputable.
    – EarlGrey
    Commented Jan 6, 2023 at 14:30
  • 6
    Their business model is set up to collect $ from academics as efficiently as possible. To this end, MDPI's modus operandi is simply to obtain the minimal peer review evidence that makes them sufficiently reputable in order to collect $. This means they are not interested in quality review. They just want to use a reviewer's name to validate their process. Commented Jan 7, 2023 at 5:52

3 Answers 3

14

Just a personal remark (that to some extent answers the question, for me personally). I usually have many things to do and am asked to review regularly, by several journals. I will strictly do reviews on a first come first serve basis. I will not be hassled by tight deadlines, and for sure a tighter deadline will not make me prioritise a review. Furthermore I decline as a matter of principle all requests that give me three weeks or less time, as this will not normally work according to the first come first serve rule.

More radically, as people know how busy everyone is, I think that if somebody sets a review deadline at less than three weeks, they implicitly communicate that they don't want a thorough high quality review, and consequently I won't review for them. This rules out reviewing for MDPI as far as I know them, but note that this is not exclusive to MDPI, and their reputation is not the issue here. However I'm not surprised in the least that they get crappy reviews given the time frame in which they're working.

I do know that high publication and reviewing speed is seen as desirable by many, and if somebody else is able to produce a good quality review in 10 days, fine by me. Still I value quality over speed, and quality takes time in my world.

Later addition: Once more not exclusive to MDPI, but... the problem with commercial publishers who use the model that the author and not the reader pays is that they have an economical incentive to accept papers. This will often play out as accepting papers more easily, and being generally more positive than critical reviewers, including potentially ignoring reject recommendations, although this will depend on the individual editor and journal. Some may be concerned enough about the scientific reputation of their journal to not accept all too bad stuff, others may accept far more than a competent and critical reviewer thinks they should.

1
  • Great sum-up of the issue I have with publishers. I wonder why the publishers take forever to post-process / edit the manuscripts, often with horrible results, while they almost exclusively try to save time on the peer-review process. Additionally, I wonder why I still have to spend a huge amount of time preparing the figures: the publishers could have/provide great tools (something like "I provide you a dummy but representative dataset, you prepare many figures, I pick the one I like"). So many questions to the publishers, but they don't care, they still get 40% profit margin.
    – EarlGrey
    Commented Jan 12, 2023 at 16:29
9

I both reviewed for and published with MDPI multiple times. My experience is in line with what is often written here: MDPI will simply look for other reviewers if they dislike your peer review report. A few months ago, I reviewed a seriously flawed paper in my field (nutrition). The authors completely misinterpreted their data. Reviewer #1 asked for minor revisions while I could not recommend the paper for publication (based on the flawed methodology). They asked for a third reviewer who finally approved the paper. It is now published - review reports did no not appear because the authors did not select open peer review.

In 2022, I submitted 3 papers to MDPI. One to Nutrients, which is one of their flagship journals. Two (well-known reviewers) asked for minor revisions which made me skeptical... In this case, peer review was open. In another instance, the same happened. A paper that has been rejected by other Q1 journals underwent minor revisions and was published in 21 days with MDPI. Finally, I had a paper with 3 reviewers that asked for major revisions. After 3 rounds, all reviewers were satisfied and indicated that the paper is suitable for publication. MDPI rejected it and invited me to re-submit it to another MDPI journal.

For me, MDPI is gambling. It is not transparent and I would only submit papers to MDPI that you may not publish elsewhere. I am honest about that. As for peer reviewing, I only review articles for MDPI from people I know. The constant pressuring for deadlines makes me uncomfortable. 5 days for a major revision is simply not enough in my opinion.

6
  • 1
    MDPI will simply look for other reviewers if they dislike your peer review report. I have to downvote this is incorrect: it's simply implausible that MDPI can do this and still keep up their blazing-fast review times. Consider, if each reviewer invitation asks for a review in 10 days, and the first reviewer takes 10 days to return a report that MDPI dislikes, then the earliest they can get another reviewer report is 20 days after submission. This is simply too late even by MDPI's timeline.
    – Allure
    Commented Jan 9, 2023 at 1:27
  • As far as I can tell, MDPI solicits three reviewer reports as the standard operating procedure. Chances are the other two reviewers simply did not read the paper in as much detail as you did (or did read the paper in detail but didn't have more recommendations), which seems to mesh with your case 2 - the reviewers are well-known and still recommended only minor revisions.
    – Allure
    Commented Jan 9, 2023 at 1:31
  • 1
    Finally, about this: After 3 rounds, all reviewers were satisfied and indicated that the paper is suitable for publication. MDPI rejected it and invited me to re-submit it to another MDPI journal. Do you have any indication that it was MDPI that rejected it and not the editorial board member handling your paper?
    – Allure
    Commented Jan 9, 2023 at 1:39
  • It depends on the journal but some invite "only" 2 reviewers. I had manuscripts accepted based on 2 review reports. 3 reports are not always required. For the last point, you are right. I cannot tell whether it was MDPI or the editorial board to reject our paper.
    – Dr.M
    Commented Jan 9, 2023 at 10:15
  • I think the target is 3 reports, but sometimes it's not possible and the journal has to make do with 2. Examples are if a reviewer agrees to review but does not submit a review, or if one simply cannot find three reviewers, to the point where the first reviews are already submitted but there is still a third missing. MDPI certainly does not invite only two reviewers - they invite many, many more.
    – Allure
    Commented Jan 9, 2023 at 10:24
7

Disclaimer, I work at MDPI as of time of writing. I also have had extensive experience in academic publishing, albeit for other publishers.

The question of whether reviews at MDPI are worse than at other publishers is difficult to answer, because to write a full answer one needs access to every journal's & every article's reviews. I don't have that kind of birds-eye view of the operations. Individual editors see only a subset of their journal's articles. More senior editors can see all the journal's articles, but there are a lot of them, so they are not likely to check every article's reviews. I'll still write an answer based on the reviews I've seen firsthand. The following numbers are for the journals I work with. Because different editors do things differently, what I see might not apply to the journals you see.

For the reviews I have seen firsthand (most of these papers are still under review, hence the relatively low number of reviews per paper):

  1. Number of papers: 11
  2. Number of reviews: 15 (all written by different reviewers)
  3. Number of reviews that include detailed comments: 10
  4. Number of reviews that do not include detailed comments: 5

Definition of "detailed" I'm using is if the review is >2 paragraphs in length, after excluding requests to cite X paper or simple English fixes. This definition is not ideal, but it's simple, and does not require subjective judgment.

Several of the reviews with detailed comments are extremely detailed reviews, spanning several pages, with two reviewers running their own calculations and including those calculations in the review (!). Among the reviews that do not include detailed comments, there was one review that basically asked for some citations without saying much else. The other reviewers still clearly read the paper, but didn't have much to say.

Compared to the reviews I saw at other publishers, broadly speaking, these reviews are excellent.

However, I have several comments about these numbers:

Of the 255 papers that underwent the entire peer review process to acceptance or rejection, about 60% of the final decisions occurred with no sign of actual peer review. ... Only 36 submissions generated review comments recognizing any of the paper's scientific problems.

So about 40% of 255 papers (=102) were peer reviewed, of those 36 of 102 (~35%) noticed the paper's scientific problems. In other words, about 65% of peer reviews did not notice the issues. Even if some of these reviews are themselves faked, the numbers are large enough that it seems likely that some reviewers simply didn't notice or care enough about the problems. This would mesh closely with my own experience. I clearly remember one Distinguished Professor at a major R1 university in the US writing a review that said nothing except "I recommend revision, and you should cite the following papers all written by me", so seniority & publishing record doesn't preclude a reviewer from writing poor reviews.

  • It should be obvious that if the editor reaches a reviewer who is very familiar with the topic of the paper, then it's more probable that the review will be good. Conversely, if the reviewer is not really familiar with the topic, they more likely to either write a poor review, unless they "get familiar" with the topic (which is a time-consuming process).

Since the MDPI standard operating procedure is to have MDPI staff identifying and inviting reviewers, and since the processes at MDPI are so accelerated (we request reviews in <2 weeks), I think it is relatively likely that MDPI reaches reviewers who aren't the best-qualified to review. In theory, these reviewers should decline to review the article, but some number surely accept anyway and then write a poor review. That could explain what you're seeing. If this is indeed the explanation, it'd indicate the editor(s) you interacted with aren't very good (whose name is on the emails inviting you to review?).

  • As I wrote above, compared to the reviews I saw at other publishers, these reviews are excellent. However, this could also be explained by me becoming a better editor (which definitely happened - my current journals are closer to my expertise, and I have been putting in a lot more effort than I used to when selecting reviewers).

For the record of the 15 reviews above, 3 of them were by reviewers not invited by me. Of those three reviews, one is very detailed (this review is one of the two where the reviewers did their own calculations); the other two are not.

  • MDPI allows reviewers to volunteer. One of these papers had three volunteer reviewers, but all three were plainly unsuited to reviewing it (no relevant expertise), so they would likely have written poor reviews too. I don't know why they volunteered. I declined all of them, but another editor might not have, which would probably lead to what you saw.

Finally: for the question "should you bother?" I think this linked question is highly relevant. Ultimately it's up to you, and you should not feel pressured to review. There is one thing that I do wonder about though: if you are the only reviewer for a manuscript, would you feel troubled? If yes, then that seems problematic for a lot of journals (I've written some papers that received had only one reviewer myself). If not, then what the other reviewers do doesn't seem like a major concern, and certainly a less important one than whether the authors take your comments seriously.

Update: I've since had the chance to look at more papers handled by other editors. I cannot say more right now, however. Maybe in a few months.

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .