28

I'm doing a review of a paper (very well written, I think) for a journal with a fairly good reputation.

This is my first review, and I have a question regarding data availability which is, according to the journal policy, a necessary condition to publish:

The data, code or other digital research materials must be publicly accessible and clearly indicated as such.

The problem is that the dataset used in the analysis can only be retrieved under the payment of a fee from a link provided by the authors.

How should I behave in this case?

The journal asked me if data was accessible and if it was adequate to allow replication. The honest answer is "no", but the authors could not have done otherwise, since the data has a license and is not free to share.

Improve this question
3
  • 14
    Actually it is not completely clear that "the honest answer is no." The policy is probably intended to require that the data are available for free, but it doesn't say that. I think you should ask the journal to clarify: do the data have to be available for free, or is being available for a fee sufficient?
    – toby544
    Commented Jan 5, 2022 at 21:28
  • 14
    must be publicly accessible, doesn't mean it is free. As a side note, the GPL license requires source code to be provided to any binary user that asked and a fee is allowed although it is required that fee to be something reasonable to cover the expenses of posting and data storage device. For sure journal has to clarify what counts as publicly accessible.
    – akostadinov
    Commented Jan 5, 2022 at 22:31
  • Have you asked the author (via the editor, if reviewing is being done blind) to purchase another copy of the data specifically for the reviewer(s)?
    – TylerH
    Commented Jan 6, 2022 at 19:41

3 Answers 3

Reset to default
22

You asked, "How should I behave in this case?" Concerning completing the form, the answer is straightforward: No, the data is not freely available.

However, I suppose that your question is asking more than that. If your concern is that you, as a reviewer, do not have sufficient material to adequately review the article according to the journal's standards (that is, you do not have access to the data), then you should email the editor who assigned the review to you and explain the situation. Either they will tell you to proceed with the review taking the authors' analysis in good faith, or they may contact the authors to ask them to provide the data as a condition to continue the review. I suspect that the editor would most likely just ask you to proceed with the review, but I do not know the journal, so they might actually ask the authors to comply with their stated policy.

Improve this answer
11
  • 18
    "publicly accessible" is not the same as "freely available".
    – Cris Luengo
    Commented Jan 6, 2022 at 18:05
  • 4
    "[take] the authors' analysis in good faith" I thought the point of peer review before publication is precisely so that we don't have to just trust that a given paper is correct/true.
    – TylerH
    Commented Jan 6, 2022 at 19:40
  • 3
    @J... Just like museums have the mission to make collections accessible to the public, but nobody bats an eye when they have to pay an admission fee.
    – Cris Luengo
    Commented Jan 7, 2022 at 17:51
  • 3
    @TylerH Maybe in some fields, but in my field (physics), it is definitely not expected that reviewers, or anyone, will reproduce the analysis, when usually a paper is the result of years of analysis by a large team. Peer review is more to verify that the steps the authors claimed to use are appropriate for the type of analysis presented.
    – thegreatemu
    Commented Jan 7, 2022 at 18:14
  • 2
    @TylerH I think that's explicitly not what it's about here. There is a strict journal requirement that the data be publicly available. I expect a reviewer's responsibilities to include ensuring the data can indeed be accessed with the provided method and that it follows expectations regarding formatting, documentation, etc. I find it highly unlikely that most reviewers have the time to reproduce enough of the work to "determine if something was done incorrectly."
    – thegreatemu
    Commented Jan 7, 2022 at 21:18
43

The journal asked me if data are accessible and if are adequate to allow replication.

That is easy to answer:

"The data is only accessible if a fee is payed as mandated by the data's license. I am therefore not able to assess if it is adequate to allow replication."

If you can only select "yes/no", select "no" and put the explanation in a comment to the editor.

Improve this answer
4
  • There is also a third option N/A (which I suppose means "not sure"). I think I will go for this and put a comment as you suggested
    – AbateFaria
    Commented Jan 5, 2022 at 9:46
  • 63
    I think "no" is the better answer - "N/A" means "not applicable", which would be the case if the paper used no sources of data (e.g. it was a commentary). The true answer here is "no" because you can't access the data. But don't worry too much about it - as long as the editor reads your note it'll be fine.
    – Rdd
    Commented Jan 5, 2022 at 10:27
  • 1
    I am therefore not able Actually, OP is not willing to assess replication, if doing so involves paying a fee to access the data. That's totally reasonable, of course, but it's not technically "not able".
    – tbrookside
    Commented Jan 7, 2022 at 13:19
  • 4
    @tbrookside That's nitpicking. If OP has no funding to pay the fee (and personal funds don't count), they are "not able". Data behind a pay-wall does not count as "accessible".
    – user9482
    Commented Jan 7, 2022 at 14:51
7

I doubt this is a problem. I suggest shooting an email to the editor of the journal with this question. It might be the case that the author(s) will be required to provide a local (downloadable) copy of the dataset that they worked with upon approval for publication (supplementary materials). Even if that's not the possible, I wouldn't necessarily interpret "publicly accessible" as "free."

It would also be helpful to know what type or journal / research project / data we're talking about here. When the authors conducted the analysis, did they do so on a local file or through something like an R server with restricted access? If they performed analysis offline, then they should have no problem providing an anonymized copy.

Improve this answer
3
  • 1
    If the original data is behind a fee-wall, I doubt the authors have a license to redistribute it in any way, not matter how they analyzed it.
    – Azor Ahai -him-
    Commented Jan 6, 2022 at 19:19
  • Yeah could be. It's not really clear at this point where the data came from, so we're all speculating here. Still I doubt the intention of the journal with this rule is to prevent the publication of good work based on data that are behind a paywall.
    – paul sparks
    Commented Jan 7, 2022 at 20:06
  • I would question whether it was good work, if the data necessary to evaluate it is inaccessible. Sounds like a scam. Like all of the ads for good health where they tell you about all of the great results from their method/product, and if you pay $99.99 they will tell you what their product/method is.
    – jmoreno
    Commented Jan 8, 2022 at 14:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .