42

I am in an interesting situation, I have a masters degree in electrical engineering but have been working as a software engineer for about 10 years. I have a lot of interest in doing peer reviews and contributing to research part-time in computer science. But I find it very hard to convince anyone to let me review their work.

My question is, is it possible to do so while working in a day job? If so, how can one go about signing up to do peer reviews? All the journals seems to want to see credibility before you can peer-review any articles or papers.

How can one get started midway through their work career?

  • 45
    I don't want to come off as dismissive and your willingness to contribute to science is honourable. But to give you some perspective: As an author, I absolutely would not want my paper to be reviewed by a reviewer who doesn't have significant experience with having their own papers reviewed. – lighthouse keeper Dec 8 '17 at 22:38
  • 10
    I may be misunderstanding you here, but it sounds as if you're trying to do peer review as an extra "job". It is not a job - in particular, it is not paid work. Rather, it is just a part of the full-time job of being a researcher. A part that cannot be dissociated from the rest, because you can't have the skills for peer review if you don't write papers. – Vincent Fourmond Dec 9 '17 at 0:16
  • 1
    No, I am interested in research. Not looking for paid work. – xoail Dec 9 '17 at 1:23
  • @lighthousekeeper it makes sense. I see why it could get harder to just try to review work. – xoail Dec 9 '17 at 1:23
  • 55
    To be a peer reviewer, you have to be a peer - if no one is reviewing your articles then you're not really a peer. First step: start submitting research to be published. – curiousdannii Dec 9 '17 at 2:04

11 Answers 11

69

As an editor, when I need peer-reviewers, if I don't know someone with expertise in the topic at hand, I look to see who has published peer-reviewed work in a similar area recently. No-one signs up to do reviews, it's up to me to go out and find them. Which means, if you haven't published peer-reviewed work recently, you aren't likely to get any invitations.

So, in my case, if you want to do peer-reviews, the first step is publishing something in a peer-reviewed journal. Otherwise, no one will know you're there.

This doesn't mean someone who isn't publishing is never qualified to provide a peer review. But if I don't know you, and you haven't published anything yourself, I have no way to evaluate if you are qualified to provide a good review.

Another thing to keep in mind is that the distinction between an interesting paper, and an interesting publishable paper, may not be clear to someone who isn't actively pursuing academic work in the area.

  • 3
    This is insightful. But in my case, it sounds like a catch 22. I need to find some editors to trust me but they wont until I have credibility and publishing a paper is incredibly hard especially since I no longer attend school. – xoail Dec 8 '17 at 17:04
  • 39
    @xoail Right. But "peer review" is quite suggestive: it is intended as a type of quality control among peers. If you are not publishing papers yourself and don't intend to, then you must agree that it would be odd to include you as "peer reviewer". I think what you are envisioning ("professional reviewers" that don't necessarily publish themselves being part of the reviewing process) could be possible, it is just not how academia has arranged itself to deal with the problem of needing reviewers for publishing papers. (And I'm not saying the current system is perfect either.) – Earthliŋ Dec 8 '17 at 18:04
  • 42
    @xoail A catch-22 is when you're required to have done A in order to do A. Here you're required to have done A (publish) in order to do B (review), and there is no requirement to have done A to do A. – E.P. Dec 8 '17 at 18:48
  • 14
    you don't have to be in academia to publish. It is a lot harder to do so, but that's not unique to academia. It's harder to participate in any professional activity when it's not your profession. – Tyler Dec 8 '17 at 19:29
  • 5
    @xoail There's several threads on publishing from outside academia on this site, academia.stackexchange.com/q/3010, academia.stackexchange.com/q/3171, academia.stackexchange.com/q/27193, academia.stackexchange.com/q/385, and their many Linked questions (on the sidebar on the right) among them. – E.P. Dec 8 '17 at 20:14
16

In addition to Thomas’ answer, peer review (in my experience) is somewhat of an “invitation only” closed club: journal editors need to know your name, and they do so because you have published in the relevant (sub-)field. And even then, editors tend to favour senior researchers or people they otherwise trust.

Case in point, I’ve published several papers during my PhD and (co-authored) during my postdoc. Yet the only peer review invitations I have received so far have been due to a specific recommendation from former group leaders to the editors.

This will probably vary across fields. But the fundamental principle is the same: editors need to put some level of trust into the competence of the reviewer. And since the research happens at the cutting edge (otherwise it wouldn’t be very interesting research), the reviewers are often chosen from a fairly small pool of people known to work at that cutting edge. Unless you have recently done outstanding, publicly known work in a field of active research, it’s unlikely that an editor would consider you as a reviewer. Merely having proficiency in already established methods is not useful.

  • 1
    Not necessarily. I've received requests from multiple journals in which I haven't published in. You can also be asked to review if your paper in another journal is a close match to the content of a paper they have received. – aeismail Dec 8 '17 at 16:43
  • 2
    @aeismail So have I. Editors talk to each other (and to senior researchers). I wasn’t trying to say that you need to have published in that journal to be invited to review there. I’ll clarify this. (In fact you’re right, my phrasing was simply wrong. No idea how that happened.) – Konrad Rudolph Dec 8 '17 at 16:44
  • @aeismail you mean publishers? bcs it is normal to be invited by different journals, but is it normal to be invited from different publishers? – SSimon Dec 9 '17 at 4:01
  • @SSimon: I have received invitations from both journals and publishers that I have not submitted papers to. – aeismail Dec 9 '17 at 4:03
  • 2
    @SSimon Finding the email address of a person in academia is usually pretty easy. Look at their web site, or their department's web site. If an editor really has difficulty finding a potential reviewer's email address, (s)he would probably just look for a different potential reviewer. – Andreas Blass Dec 9 '17 at 4:16
12

In general, and I accept that there may be some exceptional sub-field I do not know about, peer reviews in Computer Science are conducted by researchers. You need experience doing research (viz. writing papers) in order to understand the quality of what you are reading. Most people are introduced to peer review during their PhD studentship when a program committee member delegates one or more of their papers to a PhD student, typically their own student. Once someone has a PhD, they may then be invited as a program committee member or a journal reviewer by the program chair/journal editor.

  • 1
    Nice answer! I find a little bit frightening, such a power in the hands of subjectively selected, anonymous people. – peterh Dec 8 '17 at 15:49
  • 1
    @peterh Yes, sometimes reviewer politics happen, sadly. : ( – rlb.usa Dec 8 '17 at 16:34
  • Reviewers may or may not be anonymous, in my experience most are not. – E. Douglas Jensen Dec 14 '17 at 1:24
9

I have 25 years experience as an industrial researcher with an exceedingly short publication list (2 papers written when I was in school in the late '80s) and have been an active reviewer for years. So emphatically NO! you do not have to be active in research in order to be a peer reviewer. It's a chronic complaint of editors and reviewers alike of how overworked they are, and yet they completely overlook the incredibly large number of people with advanced degrees that are not in academia. Or worse yet, insist that they are unqualified...

So go ahead and contact editors. It might be wise to temper expectations (don't bother with Nature or Science for instance), but start asking. And then once you do start reviewing, be sure to record them with Publons. That will give you credibility so that you can work your way up the ladder.

  • This is a great response. I have a PhD so am used to the peer review process. I like to contribute as much as possible but at times it can be taxing in the least. I don't see why someone with a masters degree and 10 years experience in industry wouldn't be able to appropriately review research. Given that that the criteria comes down to approximately 1) is it novel? 2) Is it good science? 3) Does it contribute knowledge to the community?, these can be assessed by anyone who is informed, irrespective of their academic credentials. A lot of the answers here are far too negative! – Nathan Thomas Dec 10 '17 at 18:51
  • 1
    There's a pretty big difference between a short pub list and no pub list. Since the OP doesn't list publication experience, unless there is some sort of mentoring in place for the new reviewer, refereeing can become problematic. – Scott Seidman Dec 10 '17 at 20:33
  • Many journals have tried to combat this directly in recent years by offering reviewers a rating system (e.g., how is X on a scale from 1 - 5). In the instructions provided by the journal, they ask the reviewer a few basic questions to help them focus their review. Often the only bit left to do is to review the details of the method - I think someone with the OPs experience could check the technical vigour of the work. The rest is almost 'paint by numbers' – Nathan Thomas Dec 12 '17 at 1:16
  • As an ex-Editor of a highly prestigious CS journal, I was always in need of reviewers (as all Editors are). But I would have never signed up someone as described by doctorjay (no offense intended)--mainly because that risks strong negative feedback from the submission authors (reviewers usually are not anonymous). I had enough negative feedback from submission authors about reviews by prominent researcher reviewers. – E. Douglas Jensen Dec 14 '17 at 1:22
  • @E.DouglasJensen that's a fair comment if the reviewers are not anonymous in CS. I am used to anonymity so having negative feedback from authors due to this never occurred to me. – Nathan Thomas Dec 15 '17 at 0:25
7

One of the main issues (which I think has not yet been mentioned in other answers) is that to be a good reviewer you need to be current on what is going on in the field. So if you're not an active researcher, it will be hard for you to properly evaluate new work in context.

So instead of looking to get more into academia starting by doing peer reviews, perhaps consider getting involved with research. You could either start doing things that are of interest to yourself on your own and then contact experts when you make some progress to give feedback, or try reaching out to people/groups who you'd be interested in working with (preferably someone you know, like former professors, as "cold calling" has low response rates).

  • 1
    So if you're not an active researcher, it will be hard for you to properly evaluate new work in context. adding to this issue is (in my own experience) the fact of a much less generous access to published material when pursuing a job outside academia. Not every field uses arXiv-like pre-prints, open access - though increasingly used - is all but comprehensive, and enterprises tend the spend much less on pay-walled articles than research institutions and their affiliated libraries. – Ghanima Dec 9 '17 at 2:47
7

It's usually the opposite problem -- once you start publishing, you end up on the page editors' lists of possible reviewers, and never get off of those lists again until you die.

Given that, the real requirement for referees is that they publish in the field they review. If you haven't done so, you will have a hard time convincing editors of your qualifications.

6

To answer your questions:

  • is it possible to do so while working in a day job?:

Yes. All peer-reviewers are done by people who have other jobs. The only caveat is to make sure you employer does not prohibit it (I do not know why they would, but there might be some (real or perceived) concern over intellectual property rights).

  • If so, how can one go about signing up to do peer reviews? All the journals seems to want to see credibility before you can peer-review any articles or papers.

Other answers have described the traditional approach for being a reviewer. I have used another method for one journal I wanted to review for: Network and volunteer to a reviewer.

Specifically, I had a mentor tell me to peer review as a way to enhance my standing within a professional society. He told me to email a former professor of mine who was an editor for a journal and offer to review for him. That got my foot in the door as a reviewer with the journal. In your case, do any of your MS faculty server as editors? Or, do you know any other editors? Perhaps you could reach out to them to be a reviewer. Send them a short note and volunteer to be a reviewer.

  • How can one get started midway through their work career?

If you're able to get your foot in the door, do a good job. Be prompt, be professional, and be rigorous.

  • 1
    Let me put some emphasis that the main - or even the only relevant - point is that that he was referred to the editor by someone already known, and assumed to be competent to the editor. Only that established the assumption that he would be a valuable reviewer - concluded from the professors assumption of the same. The editor may have based his decision on this assumption alone, without caring about any other indications of the reviewers competence at all, and rightly so. Just finding an editor and asking him independently is totally different. – Volker Siegel Dec 8 '17 at 22:02
  • @VolkerSiegel I'm sorry, but I do not understand the point of your comment. Would you please try rewording it. I suggesting the OP contact people he knows who are editors. Your comment suggests am telling him to cold call people: i.e. "Just finding an editor and asking him independently is totally different." – Richard Erickson Dec 8 '17 at 22:40
  • I have written an answer, making it more clear, I hope. Please ask again if the answer does not clarify it. I read it as suggesting cold calling known editors, right? – Volker Siegel Dec 9 '17 at 10:57
2

Very long shot.

I have on occasion asked an editor who sent me a paper to review if I could pass the task along to a junior colleague (who had no reputation yet). The editor trusted my judgement, my colleagues did the job well and so got a foot in the door.

Perhaps contacting one of your former professors would work this way for you.

  • Oh, that is a good example of the situation described in my answer, published just three minutes earlier! – Volker Siegel Dec 8 '17 at 22:18
2

For some journals, you may register as a peer reviewer through a website. I recall certain elsevier journals, though the publisher may be considered expensive/restrictive for the research community. (In fact, a search for "register as a reviewer" in a web search engine does provide some entry points.) Many editors are suffering because they don't find reviewers for months! You simply have to write an e-mail to a managing editor (or to all the editors) of a journal. No guarantee you'll get a paper if you contact just the main editor of a single journal, but my best bet is that there is a pretty good chance if you contact all the editors of, say, 100 journals.

However, doing a proper peer review of a serious and well-written 50-page paper takes time. Up to several months, if you have to acquire knowledge beforehand and really go into detail. For such papers, during reading, you won't get enough grounds to stop and reject. Don't expect it to be doable in the evenings.

  • I definitely do not plan to do it over the evening and understand that some of them require good time and dedication that could last potentially months. – xoail Dec 8 '17 at 16:56
  • @xoail Good! Just wanted to make you aware of that. – Leon Meier Dec 8 '17 at 16:59
  • 1
    Some journals (I can confirm this is the case for American Physical Society journals) have a form where you can recommend others as reviewers. This may or may not be the case in OP's field; if it is, some networking with contacts who are still in academia could do a lot to get OP's name into the relevant rolodexes. – E.P. Dec 8 '17 at 18:52
0

I can think of one counter-example to my experience as an ex-Editor of a prestigious CS journal who would never have reviewers such as the OP. That counter-example was a scientist at the (U.S.) Naval Research Laboratory. He never did any direct research or publish papers. What he did do intensively for a couple of decades was to follow certain topics of interest to the NRL in the best scientific research journals and conference proceedings, and retained and assimilated it. He had a prodigious memory and ability to relate to what was in his memory. I would not have hesitated to have him be a reviewer for these topics he followed.

-1

Another answer describes an example about someone without the qualification. In most answers he was successful in becoming a reviewer. That was after a professor who knew him provided a reference.

Let me put some emphasis that the main point is that that he was referred to the editor by someone already known, and assumed to be competent to the editor.

Only that established the assumption that he would be a valuable reviewer - concluded from the professors assumption of the same. The editor may have based his decision on this assumption alone, without caring about any other indications of the reviewers competence at all, and rightly so.

An example where this would make perfect sense is when the professor knew that the reviewer is qualified to publish leading edge papers of the subfield, but did not publish on it due to some unrelated reason. Say a psychological condition of chronic paper writing phobia.


This may seem exceedingly rare to happen in the real world. But since the answer of Ethan Bolker provided a second real world example, in addition the one used in the first paragraph, just three minutes after I posted this answer, I allow myself to hope there is some non-zero credibility to it.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.