42

I am in an interesting situation, I have a masters degree in electrical engineering but have been working as a software engineer for about 10 years. I have a lot of interest in doing peer reviews and contributing to research part-time in computer science. But I find it very hard to convince anyone to let me review their work.

My question is, is it possible to do so while working in a day job? If so, how can one go about signing up to do peer reviews? All the journals seems to want to see credibility before you can peer-review any articles or papers.

How can one get started midway through their work career?

Improve this question
5
  • 47
    I don't want to come off as dismissive and your willingness to contribute to science is honourable. But to give you some perspective: As an author, I absolutely would not want my paper to be reviewed by a reviewer who doesn't have significant experience with having their own papers reviewed.
    – lighthouse keeper
    Dec 8, 2017 at 22:38
  • 11
    I may be misunderstanding you here, but it sounds as if you're trying to do peer review as an extra "job". It is not a job - in particular, it is not paid work. Rather, it is just a part of the full-time job of being a researcher. A part that cannot be dissociated from the rest, because you can't have the skills for peer review if you don't write papers.
    – Vincent Fourmond
    Dec 9, 2017 at 0:16
  • 1
    No, I am interested in research. Not looking for paid work.
    – xoail
    Dec 9, 2017 at 1:23
  • @lighthousekeeper it makes sense. I see why it could get harder to just try to review work.
    – xoail
    Dec 9, 2017 at 1:23
  • 57
    To be a peer reviewer, you have to be a peer - if no one is reviewing your articles then you're not really a peer. First step: start submitting research to be published.
    – curiousdannii
    Dec 9, 2017 at 2:04

11 Answers 11

Reset to default
71

As an editor, when I need peer-reviewers, if I don't know someone with expertise in the topic at hand, I look to see who has published peer-reviewed work in a similar area recently. No-one signs up to do reviews, it's up to me to go out and find them. Which means, if you haven't published peer-reviewed work recently, you aren't likely to get any invitations.

So, in my case, if you want to do peer-reviews, the first step is publishing something in a peer-reviewed journal. Otherwise, no one will know you're there.

This doesn't mean someone who isn't publishing is never qualified to provide a peer review. But if I don't know you, and you haven't published anything yourself, I have no way to evaluate if you are qualified to provide a good review.

Another thing to keep in mind is that the distinction between an interesting paper, and an interesting publishable paper, may not be clear to someone who isn't actively pursuing academic work in the area.

Improve this answer
6
  • 3
    This is insightful. But in my case, it sounds like a catch 22. I need to find some editors to trust me but they wont until I have credibility and publishing a paper is incredibly hard especially since I no longer attend school.
    – xoail
    Dec 8, 2017 at 17:04
  • 39
    @xoail Right. But "peer review" is quite suggestive: it is intended as a type of quality control among peers. If you are not publishing papers yourself and don't intend to, then you must agree that it would be odd to include you as "peer reviewer". I think what you are envisioning ("professional reviewers" that don't necessarily publish themselves being part of the reviewing process) could be possible, it is just not how academia has arranged itself to deal with the problem of needing reviewers for publishing papers. (And I'm not saying the current system is perfect either.)
    – Earthliŋ
    Dec 8, 2017 at 18:04
  • 43
    @xoail A catch-22 is when you're required to have done A in order to do A. Here you're required to have done A (publish) in order to do B (review), and there is no requirement to have done A to do A.
    – E.P.
    Dec 8, 2017 at 18:48
  • 14
    you don't have to be in academia to publish. It is a lot harder to do so, but that's not unique to academia. It's harder to participate in any professional activity when it's not your profession.
    – Tyler
    Dec 8, 2017 at 19:29
  • 5
    @xoail There's several threads on publishing from outside academia on this site, academia.stackexchange.com/q/3010, academia.stackexchange.com/q/3171, academia.stackexchange.com/q/27193, academia.stackexchange.com/q/385, and their many Linked questions (on the sidebar on the right) among them.
    – E.P.
    Dec 8, 2017 at 20:14
16

In addition to Thomas’ answer, peer review (in my experience) is somewhat of an “invitation only” closed club: journal editors need to know your name, and they do so because you have published in the relevant (sub-)field. And even then, editors tend to favour senior researchers or people they otherwise trust.

Case in point, I’ve published several papers during my PhD and (co-authored) during my postdoc. Yet the only peer review invitations I have received so far have been due to a specific recommendation from former group leaders to the editors.

This will probably vary across fields. But the fundamental principle is the same: editors need to put some level of trust into the competence of the reviewer. And since the research happens at the cutting edge (otherwise it wouldn’t be very interesting research), the reviewers are often chosen from a fairly small pool of people known to work at that cutting edge. Unless you have recently done outstanding, publicly known work in a field of active research, it’s unlikely that an editor would consider you as a reviewer. Merely having proficiency in already established methods is not useful.

Improve this answer
7
  • 1
    Not necessarily. I've received requests from multiple journals in which I haven't published in. You can also be asked to review if your paper in another journal is a close match to the content of a paper they have received.
    – aeismail
    Dec 8, 2017 at 16:43
  • 2
    @aeismail So have I. Editors talk to each other (and to senior researchers). I wasn’t trying to say that you need to have published in that journal to be invited to review there. I’ll clarify this. (In fact you’re right, my phrasing was simply wrong. No idea how that happened.)
    – Konrad Rudolph
    Dec 8, 2017 at 16:44
  • @aeismail you mean publishers? bcs it is normal to be invited by different journals, but is it normal to be invited from different publishers?
    – SSimon
    Dec 9, 2017 at 4:01
  • @SSimon: I have received invitations from both journals and publishers that I have not submitted papers to.
    – aeismail
    Dec 9, 2017 at 4:03
  • 2
    @SSimon Finding the email address of a person in academia is usually pretty easy. Look at their web site, or their department's web site. If an editor really has difficulty finding a potential reviewer's email address, (s)he would probably just look for a different potential reviewer.
    – Andreas Blass
    Dec 9, 2017 at 4:16
12

In general, and I accept that there may be some exceptional sub-field I do not know about, peer reviews in Computer Science are conducted by researchers. You need experience doing research (viz. writing papers) in order to understand the quality of what you are reading. Most people are introduced to peer review during their PhD studentship when a program committee member delegates one or more of their papers to a PhD student, typically their own student. Once someone has a PhD, they may then be invited as a program committee member or a journal reviewer by the program chair/journal editor.

Improve this answer
3
  • 1
    Nice answer! I find a little bit frightening, such a power in the hands of subjectively selected, anonymous people.
    – peterh
    Dec 8, 2017 at 15:49
  • 2
    @peterh Yes, sometimes reviewer politics happen, sadly. : (
    – rlb.usa
    Dec 8, 2017 at 16:34
  • Reviewers may or may not be anonymous, in my experience most are not.
    – E. Douglas Jensen
    Dec 14, 2017 at 1:24
9

I have 25 years experience as an industrial researcher with an exceedingly short publication list (2 papers written when I was in school in the late '80s) and have been an active reviewer for years. So emphatically NO! you do not have to be active in research in order to be a peer reviewer. It's a chronic complaint of editors and reviewers alike of how overworked they are, and yet they completely overlook the incredibly large number of people with advanced degrees that are not in academia. Or worse yet, insist that they are unqualified...

So go ahead and contact editors. It might be wise to temper expectations (don't bother with Nature or Science for instance), but start asking. And then once you do start reviewing, be sure to record them with Publons. That will give you credibility so that you can work your way up the ladder.

Improve this answer
6
  • This is a great response. I have a PhD so am used to the peer review process. I like to contribute as much as possible but at times it can be taxing in the least. I don't see why someone with a masters degree and 10 years experience in industry wouldn't be able to appropriately review research. Given that that the criteria comes down to approximately 1) is it novel? 2) Is it good science? 3) Does it contribute knowledge to the community?, these can be assessed by anyone who is informed, irrespective of their academic credentials. A lot of the answers here are far too negative!
    – GeoMonkey
    Dec 10, 2017 at 18:51
  • 1
    There's a pretty big difference between a short pub list and no pub list. Since the OP doesn't list publication experience, unless there is some sort of mentoring in place for the new reviewer, refereeing can become problematic.
    – Scott Seidman
    Dec 10, 2017 at 20:33
  • Many journals have tried to combat this directly in recent years by offering reviewers a rating system (e.g., how is X on a scale from 1 - 5). In the instructions provided by the journal, they ask the reviewer a few basic questions to help them focus their review. Often the only bit left to do is to review the details of the method - I think someone with the OPs experience could check the technical vigour of the work. The rest is almost 'paint by numbers'
    – GeoMonkey
    Dec 12, 2017 at 1:16
  • As an ex-Editor of a highly prestigious CS journal, I was always in need of reviewers (as all Editors are). But I would have never signed up someone as described by doctorjay (no offense intended)--mainly because that risks strong negative feedback from the submission authors (reviewers usually are not anonymous). I had enough negative feedback from submission authors about reviews by prominent researcher reviewers.
    – E. Douglas Jensen
    Dec 14, 2017 at 1:22
  • @NathanThomas Reviewers are supposed to be anonymous but at least in certain sub-fields/topics it is often pretty easy to figure out who at least some of them are, based on their reviews.
    – E. Douglas Jensen
    Aug 29, 2019 at 19:38
8

It's usually the opposite problem -- once you start publishing, you end up on the page editors' lists of possible reviewers, and never get off of those lists again until you die.

Given that, the real requirement for referees is that they publish in the field they review. If you haven't done so, you will have a hard time convincing editors of your qualifications.

Improve this answer
7

One of the main issues (which I think has not yet been mentioned in other answers) is that to be a good reviewer you need to be current on what is going on in the field. So if you're not an active researcher, it will be hard for you to properly evaluate new work in context.

So instead of looking to get more into academia starting by doing peer reviews, perhaps consider getting involved with research. You could either start doing things that are of interest to yourself on your own and then contact experts when you make some progress to give feedback, or try reaching out to people/groups who you'd be interested in working with (preferably someone you know, like former professors, as "cold calling" has low response rates).

Improve this answer
1
  • 2
    So if you're not an active researcher, it will be hard for you to properly evaluate new work in context. adding to this issue is (in my own experience) the fact of a much less generous access to published material when pursuing a job outside academia. Not every field uses arXiv-like pre-prints, open access - though increasingly used - is all but comprehensive, and enterprises tend the spend much less on pay-walled articles than research institutions and their affiliated libraries.
    – Ghanima
    Dec 9, 2017 at 2:47
6

To answer your questions:

  • is it possible to do so while working in a day job?:

Yes. All peer-reviewers are done by people who have other jobs. The only caveat is to make sure you employer does not prohibit it (I do not know why they would, but there might be some (real or perceived) concern over intellectual property rights).

  • If so, how can one go about signing up to do peer reviews? All the journals seems to want to see credibility before you can peer-review any articles or papers.

Other answers have described the traditional approach for being a reviewer. I have used another method for one journal I wanted to review for: Network and volunteer to a reviewer.

Specifically, I had a mentor tell me to peer review as a way to enhance my standing within a professional society. He told me to email a former professor of mine who was an editor for a journal and offer to review for him. That got my foot in the door as a reviewer with the journal. In your case, do any of your MS faculty server as editors? Or, do you know any other editors? Perhaps you could reach out to them to be a reviewer. Send them a short note and volunteer to be a reviewer.

  • How can one get started midway through their work career?

If you're able to get your foot in the door, do a good job. Be prompt, be professional, and be rigorous.

Improve this answer
3
  • 1
    Let me put some emphasis that the main - or even the only relevant - point is that that he was referred to the editor by someone already known, and assumed to be competent to the editor. Only that established the assumption that he would be a valuable reviewer - concluded from the professors assumption of the same. The editor may have based his decision on this assumption alone, without caring about any other indications of the reviewers competence at all, and rightly so. Just finding an editor and asking him independently is totally different.
    – Volker Siegel
    Dec 8, 2017 at 22:02
  • @VolkerSiegel I'm sorry, but I do not understand the point of your comment. Would you please try rewording it. I suggesting the OP contact people he knows who are editors. Your comment suggests am telling him to cold call people: i.e. "Just finding an editor and asking him independently is totally different."
    – Richard Erickson
    Dec 8, 2017 at 22:40
  • I have written an answer, making it more clear, I hope. Please ask again if the answer does not clarify it. I read it as suggesting cold calling known editors, right?
    – Volker Siegel
    Dec 9, 2017 at 10:57
3

For some journals, you may register as a peer reviewer through a website. I recall certain elsevier journals, though the publisher may be considered expensive/restrictive for the research community. (In fact, a search for "register as a reviewer" in a web search engine does provide some entry points.) Many editors are suffering because they don't find reviewers for months! You simply have to write an e-mail to a managing editor (or to all the editors) of a journal. No guarantee you'll get a paper if you contact just the main editor of a single journal, but my best bet is that there is a pretty good chance if you contact all the editors of, say, 100 journals.

However, doing a proper peer review of a serious and well-written 50-page paper takes time. Up to several months, if you have to acquire knowledge beforehand and really go into detail. For such papers, during reading, you won't get enough grounds to stop and reject. Don't expect it to be doable in the evenings.

Improve this answer
3
  • I definitely do not plan to do it over the evening and understand that some of them require good time and dedication that could last potentially months.
    – xoail
    Dec 8, 2017 at 16:56
  • @xoail Good! Just wanted to make you aware of that.
    – Leon Meier
    Dec 8, 2017 at 16:59
  • 1
    Some journals (I can confirm this is the case for American Physical Society journals) have a form where you can recommend others as reviewers. This may or may not be the case in OP's field; if it is, some networking with contacts who are still in academia could do a lot to get OP's name into the relevant rolodexes.
    – E.P.
    Dec 8, 2017 at 18:52
2

Very long shot.

I have on occasion asked an editor who sent me a paper to review if I could pass the task along to a junior colleague (who had no reputation yet). The editor trusted my judgement, my colleagues did the job well and so got a foot in the door.

Perhaps contacting one of your former professors would work this way for you.

Improve this answer
1
  • Oh, that is a good example of the situation described in my answer, published just three minutes earlier!
    – Volker Siegel
    Dec 8, 2017 at 22:18
0

I can think of one counter-example to my experience as an ex-Editor of a prestigious CS journal who would never have reviewers such as the OP. That counter-example was a scientist at the (U.S.) Naval Research Laboratory. He never did any direct research or publish papers. What he did do intensively for a couple of decades was to follow certain topics of interest to the NRL in the best scientific research journals and conference proceedings, and retained and assimilated it. He had a prodigious memory and ability to relate to what was in his memory. I would not have hesitated to have him be a reviewer for these topics he followed.

Improve this answer
-1

Another answer describes an example about someone without the qualification. In most answers he was successful in becoming a reviewer. That was after a professor who knew him provided a reference.

Let me put some emphasis that the main point is that that he was referred to the editor by someone already known, and assumed to be competent to the editor.

Only that established the assumption that he would be a valuable reviewer - concluded from the professors assumption of the same. The editor may have based his decision on this assumption alone, without caring about any other indications of the reviewers competence at all, and rightly so.

An example where this would make perfect sense is when the professor knew that the reviewer is qualified to publish leading edge papers of the subfield, but did not publish on it due to some unrelated reason. Say a psychological condition of chronic paper writing phobia.


This may seem exceedingly rare to happen in the real world. But since the answer of Ethan Bolker provided a second real world example, in addition the one used in the first paragraph, just three minutes after I posted this answer, I allow myself to hope there is some non-zero credibility to it.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .