How would you decide to publish (or not) in F1000Research or other journals with alternative peer-review methods?

Question

As pointed out by altmetrics, “Peer-review has served scholarship well, but is beginning to show its age. It is slow, encourages conventionality, and fails to hold reviewers accountable.”

Not all publications are created equal. Neither are the statuses of their authors. However, senior authors in life sciences rarely publish in isolation, so some junior researcher’s neck is always on the line (one of the many catch 22’s in academic publishing, and one I hadn’t really thought about much before).

I realize this makes the question somewhat subjective, and I hope that a variety of answers can be supplied - but if your answer is similar to an existing one, perhaps a comment will serve - thank you!

Here is some more background on my question:

An excellent article by Zen Faulkes, which doesn’t mention F1000Research but does mention Proceedings of Peerage of Science , suggests that the post-publication peer-review is unstandardized. It seems that F1000Research is trying to standardize the process within their journal, rather than relying on altmetrics. I think this is a reasonable approach: as a scientist whose job is partly to network with others, you can very likely get interested parties to review your paper (if not, then try one of the many journals supporting the traditional review method). Additionally, this publisher-centric peer review will probably attract more serious reviews; who wants to post a review on some random social networking site that few people will likely read?

If someone has strong negative feelings about your paper, then they can say so. Of course, there is a lack of anonymity on F1000Research. Oh dear, now we need to be polite (Faulkes also discusses this). I know it is a very difficult thing to be polite sometimes after multiple grants or papers have been rejected, but perhaps this is a step in the right direction.

Perhaps more importantly, the reviewers may have to defend their assertions without the editor making a summary decision about whether or not the authors can reasonably address the reviewers concerns. I think this form of review removes much more bias than it introduces by keeping the conversation open. And I don’t think this means the review process will go on indefinitely; the authors no longer need satisfy every whim of the reviewers, but only those they deem important enough so that their colleagues will respect the merit and credibility of the study.

Turning to other forms of review: I also think it is unlikely that any single review model with always be best for all people and all publications.

For instance, one type of review process that would necessarily need to be pre-publication peer review is the double blind scheme, where neither reviewers nor authors are given any direct evidence of the other party’s identity. It seems that this would not only make things more fair, but also open up the reviewer pool to highly cited authors whose input is no doubt frequently valued but difficult to obtain in the traditional peer review process. Unfortunately it does not seem to be at all prevalent in the sciences, and I know of no journals supporting it.

Answer 1

How would I decide?

Funding, Readership and Tenure.

• Does the journal cost money to submit to, and if so, do I have any funding for said submission? Is there a means of waiving that fee if I don't have funding?
• Is the journal read by the people I want to read my paper? This I'd gauge by talking to my colleagues - for example, if no one in (hypothetically) infection control reads F1000Research, then I'm not publishing there, because the point of my paper is to be read and advance the field, and it can't do that if it isn't where relevant eyes will see it, whether or not the journal gets an ideological checkmark.
• Similar to readership, when this journal is looked at by a promotion committee etc., will it matter?

The latter two are, in my mind, not yet settled, and that makes me a little hesitant - bold ideological stands are for people who aren't on the job market. Beyond that, I think the primary problem with post-pub peer review is that I'm not convinced it will be at all uniformly distributed. My suspicion is that a few prominent papers will be heavily reviewed, and then the rest will have either middling, or no reviews beyond what is mandated by the journal. I'm also not convinced it's a terribly fair system - it's relatively easy to attack a post-doc or doctoral student's work, while you might hesitate to do so for Important Person In Your Field, and similarly, said Important Person may be able to be fairly rude with impunity, especially if the journals aren't terribly well known.

For what it's worth, several major journals in my field are double-blinded. I have no idea who the authors are, there are actually instructions to remove clearly identifiable information (change "Brigham-Women's Hospital" to "A major regional hospital in the Northeast United States", etc.) in the manuscript, and the reviews are anonymous unless signed.

Answer 2

Subjectively, I'd say that there are two main criteria on deciding where (and if) to publish some piece of research.

1. Does that publication "count"?

There tend to be specific criteria set by funding agencies, promotion committees or other organizations that separate 'proper' publications from worthless ones. Clearly, there must be some criteria, as the world is full of places that would publish anything, and counting everything equally is a worse option than most completely arbitrary criteria. For some areas the split might be journals vs conferences/etc; for some places the criteria might be [lack of] peer review; for my purposes the criteria is indexing in Web of science or SCOPUS.

2. Would that publication be read by the people I want?

When academic goals involve 'advertising' a particular approach and influencing the subfield, it doesn't matter how if it's reviewed (if at all), but it matters on how well it reaches the target audience. For that purposes the best option might be, say, a short talk at some seminar which is well attended by the relevant group; which could have more impact than an article at a prestigious journal, and has the benefit of being quicker than the traditional review process.

Depending on ones particular goals, either of those goals may strongly dominate the other; however, if none of them are well met, it makes no sense to publish (at that venue) at all.

For me, it seems that any new or alternative approach is very unlikely to be useful for publishing. Alternative approaches by definition don't match the classic, estabilished ways of publishing (goal #1); and any new publication, unless estabilished by scientists in my particular narrow research area for their own needs, is unlikely to reach them (goal #2) better as any other venue , even the trivial option on simply putting it up on your own or organization webpage.