I spent a lot of time during my post-doc doing a big-data analysis using cloud credits paid for by a grant. I got the research conducted, written up, and submitted to a journal. The modeling I did is not possible to conduct outside of a cloud environment.

Now, the paper has spent a long time in review, and that grant has expired, the cloud account has been closed, and all of the data has been deleted (although I still have the code).

The reviews are mostly positive, although some of them are requesting minor tweaks to the model, which is not possible. I also really don't think that the tweaks they are suggesting would improve the model.

What I am wondering is, how candid should I be about the grant situation? How much should I focus on the science versus my excuses for why I simply can't update the model?

Should I:

  • Focus on the science and explain why I think the proposed changes to the models likely wouldn't affect the outcome?
  • Discuss the grant details and tell the reviewers that our grant/cloud access expired, and therefore I can't re-run the models?
  • Some combination, i.e., tell the reviewers that their proposed changes likely wouldn't affect the model findings, focusing on the science, but then tell the editor about the grant/cloud situation?

I'm also wondering what happens in other fields where experiments can only happen once, and it is impossible to update the findings.

  • 75
    "all of the data has been deleted" can you explain that a bit more? Very often experiments cannot easily be repeated, but loosing all the raw data is, well, really bad if you want to write a paper.
    – Karl
    Jul 27, 2022 at 22:18
  • 6
    Many journals will insist upon a data retention policy. Not having the data might remove the paper from consideration. Jul 28, 2022 at 11:58
  • 4
    Your sponsor may have such a policy as well Jul 28, 2022 at 12:00
  • 18
    "all of the data has been deleted" - are you sure about this? At least the final raw data that has been used for the ppublication definetly should've been preserved some way. Loss of such expensive data should never happen, it seems it was was very expensive to obtain it. It could be reused for further research; your grand may even require you to keep it accessible. Your problem might be bigger than a simple refusal.
    – Neinstein
    Jul 28, 2022 at 13:27
  • 5
    @JeopardyTempest if the numerical experiments can be re-run without further inputs, then those aren't data, they are intermediate values. In my view, at least, but maybe that's just a matter of terminology. Jul 29, 2022 at 12:05

3 Answers 3


A frequent response in experimental fields goes something like:

"We appreciate the reviewer's suggestion to use the 'X' technique, and do believe this could lead to an interesting extension of our current results. However, due to the unavailability of 'Y' resource, we are currently unable to undertake this study. We have, however, run a preliminary analysis* along the lines suggested, and found that it does not alter our main results significantly.

...some details/description of the analysis "

*The 'preliminary analysis' is often just a theoretical justification or back-of-envelope estimation. Its purpose is to establish that the suggested changes won't drastically affect your main claim(s). Including this in the response shows that you've genuinely tried to incorporate feedback within the available resources.

Be mindful of the conventions in your field though; @Kimball and @Anyon have pointed out that 'preliminary analysis' would convey something quite significant in their fields. If so, the response above may come across as flippant/disrespectful, and you may consider making a weaker claim, such as 'we estimate', 'we expect', 'our initial assessment is...'

  • 7
    Coming from pure math, I would expect the phrase "preliminary analysis" to indicate more than just a back-of-the-envelope calculation. Is this phrase really widely used to merely refer to a few minutes of thought?
    – Kimball
    Jul 27, 2022 at 13:32
  • 4
    @Kimball-I understand analysis is an entire branch in math, and this usage must be a little alarming. A possible origin of the lax usage is from 'dimensional analysis', which is prevalent in some fields to make back-of-envelope estimates/justifications. Jul 27, 2022 at 14:05
  • 5
    Coming from physics, I would agree that calling a back-of-the-envelope calculation "preliminary analysis" is a bit strong. In my mind, "running a preliminary analysis" would at least involve (re)processing part of the data. Other phrases such as "we estimate" may be used instead.
    – Anyon
    Jul 28, 2022 at 15:03
  • @Anoyn- thanks, point well taken and incorporated into the answer. Jul 29, 2022 at 9:45

As an addendum to the other answers, I would also suggest including a brief note about the total computing resources (CPU time, RAM, disk space) you've spent on modeling the problem in your paper (and possibly a more detailed description of the software and cloud computing environments used e.g. in an appendix). This has several potential advantages:

  • It lets the reader (including reviewers) know that the modeling process was slow and costly, and not easily repeated with minor tweaks to the model, thus explaining why you have not explored such model variations further.

  • It provides valuable information to anyone interested in reproducing or extending your results about the effort likely involved in doing so. In particular, if your reviewers underestimated the effort, it's likely that other readers may do so as well. Letting them know that your model was expensive to run may save them from wasting time and money on a project they cannot finish, which they may thank you for later.

  • Conversely, it's possible that some other group might be able to carry out similar modeling much more easily, either because they have more money, faster computers (perhaps because they're reading your paper 10 or 20 years later) or simply more efficient modeling software. (Never discount the latter possibility, especially if you're using off-the-shelf software. It's amazing how often even minor algorithmic improvements can speed up a model by multiple orders of magnitude.) Knowing that the modeling task was challenging for you, with the software and computing resources you had, might inspire someone to write a followup paper showing how to model your system more efficiently (which, of course, would directly benefit your group too) or even to offer to collaborate with you on the problem.

In general, if an experiment you've performed was difficult to carry out for some reason, you should always make that clear in your paper — not merely to brag about doing it, or as an excuse for not doing it more times, but also as a challenge for others to find ways to do it more cheaply and efficiently. That is how methodology advances.


I would recommend stating the reasons why you don't think tweaking the model will make much difference but also mention that you no longer have access to the computing facilities necessary to re-run the model. You could also say that modifying the model could be part of future work (by yourself or others).

It's not unusual to be unable to repeat an experiment at a later date. I've done research with explosives. The test specimens, and sometimes the instrumentation, no longer exists after the test. Or at least is in very small pieces spread across a test site. Sometimes equipment is damaged and some data cannot be recovered. It costs a lot to do such tests and repeating them is simply not feasible financially.

Ultimately either the tweaks are essential and the work will be rejected, or the work is satisfactory as is and will be published (at this journal or another one).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .