Are editorial management systems helpful to journal editors?

Question

In this amusing blog post, Scott Aaronson explains why he is unwilling to deal with editorial management systems as a referee. They make you log in to some complicated system, using a password you've probably forgotten, all to provide your expertise for free. They often don't give you the sense that you are corresponding with an actual human.

And they can go hilariously wrong. In March of this year (2017) I was informed that I was supposed to have finished a referee report by May 15, 2017, that my report was therefore "-51 days late", and that I should therefore finish it as soon as possible.

From my perspective simple e-mail correspondence works better as a referee. (Many math journals make do without these systems.) However, I have never been a journal editor, and so I wonder if I am missing the point. Do editors genuinely find these systems to be more convenient for managing their workloads? Or alternatively is it publishers that prefer to use them, and thus ask the editors to go along with them?

Answer 1

I served as editor-in-chief of an IEEE magazine for 4 years and have been an associate editor on various editorial boards for the past couple of decades. For as long as I've been on edboards, I'd say they have used some flavor of online management system. Pretty much everything I'm on now, not to mention everything I submit to, has coalesced on "manuscript central" from Scholar One.

I fully agree with @Nate Eldridge's comment about the "paper" trail. That is definitely a plus, because I can look and see past correspondence -- was someone reminded at the right time, etc. And to be honest I couldn't imagine managing this offline.

That being said, I find the system (and the others I've used) frequently painful. Aside from the bug about being negative days late, my biggest problem is that the system tends not to remind anyone that a task is outstanding. In particular, as an associate editor my job is to line up the actual reviewers. In two cases recently, I discovered that I'd invited 3 people, and usually more (as people might decline), but in the end 2 from each of 2 papers had agreed to review and 1 in each case had never responded. The system sent them a reminder or two, then from what I can tell, it neither reminded them nor told me it was time to find someone else!

There are other minor difficulties, such as finding reviewers within the system rather than creating duplicate accounts for them as new reviewers, that sort of thing.

Bottom line: they serve a purpose, but they can be improved.

---

Are they helpful to journal editors?

Absolutely.

Can they be improved, especially from the standpoint of reviewers?

Absolutely.

But don't throw the baby out with the bathwater.

Answer 2

From several discussions with editors and publishers and my own experience as an author, a referee, and a OA advocate, I can infer that:

• editorial management systems can be useful to editorial boards, in particular some give the adequate warnings and reminders to editors, as Fred Douglis mentions he would have expected,

• they can be nice to authors and referees (in particular, some of them don't need every referee to log in; some use ORCID identification to have you get a unique login/password for all systems who do the same),

• most systems, especially those used by the largest, most expensive publishers are much less useful and much less nice to authors and referees than they could, probably because changing them is a huge problem when you have thousands of boards to embark in this move,

• these systems are definitely liked by large publishers, who collect data through them; in particular they collect data about who refers what, and use it to provide editors with referee suggestions. I expect them to also use it to target advertisement, to map science and try guess in which area a new journal could be launched, and probably other purposes I don't imagine,

• at least one board of a very important and quite large journal avoids the editorial management system provided by its large publisher precisely to avoid giving it all this data, so there are ways to avoid it.

Answer 3

From the point of the reviewer, it's easier to just use email, since the reviewer is usually only handling one review (at least for the journal in question). For the editor however, it doesn't work that way. The numbers quoted below are order of magnitude estimates.

Let's say you're handling ten papers, each of which needs two reviewers to proceed. Because there's a nontrivial chance an invited reviewer will decline or not respond to the invitation, you might have 3-4 reviewer invitations going for each of those submissions. That's 30-40 different invitations to monitor. In addition to that, you still have to monitor when reviews are due, if any are overdue, and whether or not to invite more reviewers for those. If you are the editor-in-chief, you have lots more papers to monitor as well as associate editors to keep track of. Without the editorial management system, keeping all these things up to date is very difficult.

There are also other administrative benefits as well: for example, to keep track of whether you can invite reviewer X or if another editor has already invited X for a different paper; to keep track of whether you've sent reviewer Y three invitations in the past 3 months and so might be taking up too much of Y's time; to keep statistics of turnaround time to quote on the journal's website; to answer author queries about when their manuscripts will be ready.

Having said all that, if you find using the editorial management system to be an unbelievable hassle, you can probably email the publisher and get them to upload your review for you. That's because based on personal experience, few reviewers request the publisher do this, so the desk editor can probably cope with this tiny bit of extra work. I found it tedious, but still, someone has to do it, and the reviewer's time is more valuable than mine. If you're doing this, be sure to mention any confidential comments you might have, and indicate whether you want to see the manuscript again if the decision is revise.

PS: Please don't everyone start emailing the publisher.