I never tried to avoid becoming a sub-reviewer, and still, this question made me wonder. How much time should I, as a PhD student, spend on a review of a paper that my advisor gave me to do? Does the answer depend on whether I'm getting credit for the review or not, that is whether I'm a sub-reviewer or not?
You should do slightly more reviewing work for the community than the community does for you. So, as a rough estimate, assuming every paper you write and submit needs three reviews and your average paper has three authors, you should review slightly more papers than you submit.
There is no way to predict in the abstract how long a paper takes to review; that varies from field to field, subfield to subfield, and even paper to paper. Answering a similar question in Theoretical Computer Science Stack Exchange, I wrote:
Expect to spend about an hour per page, mostly on internalizing the paper's results and techniques. Be pleasantly surprised when it doesn't actually take that long. (If it takes significantly less time than that, either the paper is either exceedingly elegant and well-written, you know the area extremely well, or the paper is technically shallow. Don't confuse these three possibilities.)
No, the answer does not depend on whether you get credit for the review. If you're not going to write a thorough, professional review, just say no.
Read the entire paper in one go. For a 12 page* paper, that should take you an hour at most. This is the first pass. From this you can now decide how to proceed, now that you have (quickly) covered all the sections. (edit: The goal on this first pass is not to understand all the details, but rather get an overall feeling of the paper, quality and organization of ideas)
If a paper is terribly written and difficult to understand you should reject it outright for being so. It should not take a reader 12 hours to read a 12 page paper - and indeed, many papers are badly written and do take this much time - and you are still left wondering because it was not clear.
If the basic claims seem sound and sufficiently interesting/important and the idea is clear enough to understand, then you need to dig in. Depending on the type of paper this means looking at definitions, proofs, or experimental design, hypothesis testing and the analysis of the results. Likewise for any related work you know of that should be cited (and not just your own - that is too cliche! - if you do recommend your work as a citation at least include some others!).
All along, help people out with typos and readability. As a reviewer, you are the last chance to make this a good read before it is published.
And even if you reject it, don't just shoot people down. Give them ideas about how to make the paper stronger, suggestions for different directions, etc. Remember that it is most likely some other student who will receive your review back - so try and help them make the next paper better.
(* 12 single column pages. A 6 page paper with two columns can often contain the same amount of text as a 12 page single column paper)
There is no clearcut answer to how long a review takes. I would say that the amount of time spent on it is the time you need to understand the paper and provide good advice on wether or not it is a paper that can be accepted, with some changes or as is. In general, this should take you around a full working day. How long this exactly becomes depends on:
- The level and clarity of the paper (very bad paper takes less time, mediocre paper takes more time, very good paper takes less time).
- Your familiarity with the subject (less familiar more time).
- The requirements of the journal/conference you are reviewing for, is it a full review, or just a go/no go.
It depends on how many reviews you've done and what your experience is. If you're just starting out and have never reviewed a paper before, you should expect it to take a minimum of several hours to do a decent to good review.
Your mileage may vary depending on what discipline you're in, but a good review generally will consist of:
- a summary of what the contribution of the paper was
- a brief list of 1-3 strengths and 1-3 weaknesses that the paper had
- several paragraphs explaining those strengths and weaknesses in greater detail along with constructive suggestions of what would be needed to improve the weaknesses
- a brief listing of any related work that you feel is missing from the paper
- a recommendation to the reviewing committee of whether or not you believe the paper should be accepted
- usually there is a part for a numerical evaluation of your expertise and whether or not it should be accepted
The more reviews that you do, the less time that you should need to spend on doing the review. It seems like many professors can knock out a review in an hour or less, but most of the graduate students I've seen will tend to spend at least 1-2 hours per review if they're doing a good job.
Take enough time to find most (~80%) of the important strengths and weaknesses of the paper and to describe and explain the weaknesses thoroughly and clearly but if you encounter one particular non-core aspect or section of the paper that would take excessively long to do this, just skip it and be frank: "I have not reviewed ((aspect X)) thoroughly, because my expertise in this area is too shallow."
If the editor is worth her salt, this will actually increase her confidence in the rest of your review.
All of your reviews should be careful but none of them need to be perfect.