17

I've noticed that most papers do not say explicitly (or at all) how much time did the research take. This excludes certain types of studies, such as epidemiological studies that usually say that the study's cohort was followed for some-and-some months etc., but what I mean is that in most papers there is no actual indication of how much time the study\set of experiments took. I believe this type of information could be useful for readers, especially for readers that would like to perform similar experiments for their own research.

So why is this information not required from authors?

(I realize that it's sometimes hard to pinpoint the date when a person begins and finishes a research project, but it could still be useful to give an approximation in months. This way a reader could get a better estimation if such a project is reasonable for their own research, at least from time investment considerations.)

  • 12
    The time to design & build an experiment from scratch is not necessarliy the same as that required to repeat an experiment when the issues have been solved. – Solar Mike Jul 25 '18 at 9:19
  • 2
    That's true, but it still would be helpful to know if what I'm reading took 1 year or 5 years to complete. Even rough estimations of the length of the experiment(s) can give some idea about its complexity, tediousness, etc. – Don_S Jul 25 '18 at 9:22
  • 16
    An exact amount of time (e.g., the research took 357 hours of intellectual work) is impossible to provide. On the other hand, say an idea popped out on March 2015, and the paper was submitted 2 yrs later. Did the research take 2 yrs? Not really: during that time 5 other projects were completed, there were holiday twice, time used for writing a grant proposal for the 6th project, one of the team members was hospitalized for a few months, etc. etc. So, what time interval do you want to hear in this situation (which is a pretty normal framework in research)? (...) – user68958 Jul 25 '18 at 9:24
  • 29
    Although indeed, information like we used a 2800 cores supercomputer to perform the computations, which took about a month of real time should be provided in the papers. This is an info that was completely lacking in a relevant for my work paper, that I had to e-mail the authors to even learn they used a supercomputer (and there are supercomputers and supercomputers - some with ~hundreds, some with ~millions of cores). – user68958 Jul 25 '18 at 9:34
  • 3
    In my field, someone who has a reasonable chance of completing a study successfully would know roughly how long it will take. There is usually no benefit for including this information in the manuscript. Of course, the precise duration of the actual experiment is often provided in the manuscript. – Roland Jul 25 '18 at 10:34
68

In a paper I wrote 35 years ago, I remarked that there ought to be a strong connection between a certain pair of concepts. After doing nothing with that idea for a long time, I recently worked out the connection, in a few days. I'm about to start writing this up, but first I had to check the relevant literature in order to cite it properly. One item was available only by inter-library loan, and the time spent waiting for it was roughly similar to the time spent actually proving the theorem. If someone asked me how much time it took to do this research, I could truthfully answer "a few days" and I could truthfully answer "35 years".

Furthermore, neither of those answers would be useful for my readers, because confirming my result will not involve repeating experiments but rather just reading and checking the proof in my paper. How long will that take? Maybe an hour or so for an expert in the relevant area, but much longer for someone unfamiliar with the area who needs to read a lot of prerequisite material first.

So I'd say that, at least in mathematics, information about the time required for research is likely to be meaningless and useless.

25

Because it is simply impossible to give an estimate on that time in research.
Only when you have finished the theoretical research, the thinking, the experiments, you can start planning and measuring the time needed for experiments to verify your claim, the time needed to write everything down, etc.

But for theoretical research and discovering new things, that is impossible. You can think about a problem for years and never get a solution, or you can stumble over it one day while doing something totally different and get an idea for a solution in a few days only.

Another point to add is the background of the researcher. Say two people have the same idea how to tackle an unsolved problem. One is a professor and big expert in the field, having access to a lab full of PhD-students and post docs to run experiments, the other is a Master student having, maybe, a laptop. Can you really compare the time these two need? Furthermore, when measuring the time needed to solve a problem, how do you measure the years and years of learning all the skills necessary to actually do it, how do you count all the failed attempts, all the ideas that didn't work out until you finally find one that actually works?

So my point is: It is almost impossible to measure how long it takes to get a theoretical result. You can measure things like study times, times to replicate something in the lab, running times of algorithms, etc. - but in many cases, these are either easy to deduce from the experiment description or they are given in the paper.

  • A further issue with your professor to Master student comparison is: my observation has been that more experienced people come up with ideas faster. – Oleg Lobachev Jul 25 '18 at 11:35
14

It would be difficult to pin this number down precisely. Most research projects contain an incredible number of false starts, changes in direction, and obvious-in-hindsight errors, so knowing that these authors took two years to produce a paper tells you fairly little, because

  • You may have more (or less) experience,
  • Your subjects/specimens might be different in some subtle but important way,
  • You may not need to search around for parameters that work, since they already did—or you may have to optimize something that "just works" for them.
  • and so on....

However, while this information doesn't go in the paper, you can often find it out. Researchers usually have a rough idea of what their close colleagues are working on and for how long, as in "eh, it took him about a year to get those experiments going."

An even better solution is to ask! This may get you context that would be difficult to put in a paper: no one is going to write "9 months, but the postdoc is an idiot" in a paper, but they may say "only two months, but she is amazingly good at training animals; it took the new folks more like 4-5 months and even then, the behavior hasn't totally stabilized" in conversation. You don't need to know the authors well--you could email them, ask on Twitter, or even just bump into them at a conference. People are generally happy to answer this because they either get to brag ("look how clever we are") or complain ("Yeesh, what a slog") both of which are popular conversation options. Asking may also produce an offer to collaborate, or share the actual protocols, which are probably more detailed than whatever went into the paper. So…ask!

4

The answers above do a great job of explaining why it is difficult to estimate time taken. I will add two points:

(1) Time taken would including ideation time, feasibility study time, vetting time, experiment design time (if applicable), experiment set up time (man, machine, material availability), experiment running time, data analysis time, compilation, writing and consent-acquiring time.

The majority of these are highly individual specific, and some (set up, consent) are organisation/institute/lab-dependent. These would not be useful if specified, since they are unlikely to be applicable in a different environment. The only (somewhat) universal factor is experiment running time- there is indeed a case for specifying this. In some fields, such as prediction of long-term material properties, this experiment time is crucial to mention.

(2) I don't have a citation on this, but it seems that payoff (academic or pecuniary)and material resources(man, machine, material again) dictate project choice more than time.

  • 1
    +1. Don't forget grant acquisition time. And submission-rejection-submission-waiting-for-review-revision-resubmission-rejection-banging-head-against-wall-reworking-resubmission-yet-another-rework-and-resubmission-final-acceptance-celebration time. – S. Kolassa - Reinstate Monica Jul 25 '18 at 16:02
3

Not to contradict many of the good answers above, but I'll add one more consideration, at least in the case of mathematics. Even if it were possible to give an accurate count of the number of "brain-hours" it took you to solve a problem (not including ancillary research that had to be done beforehand), you might not want to offer that information, simply out of a sense of self-consciousness. Most "average" mathematicians (including myself) are keenly aware that a problem that might take them weeks or months to solve themselves, could perfectly well have been solved in an afternoon by a mathematician of greater ability. Conversely, if you are one of those mathematicians of greater ability, mentioning the short amount time it took you to solve the problem would probably be considered bragging.

One thing however that I think mathematical writing could benefit from is in including, not the time spent, but the motivation and overall trajectory that brought them to their conclusions. Most articles in mathematics give the impression that the author just took a nap one afternoon and woke up with a fully-formed proof. It is almost always the case that the author happened upon several intermediate results that led them to their eventual conclusion, and this indeed could help to give the reader a fuller understanding of the result. But, that's just the way we write these days, and journal editors by and large don't like it when you include that kind of meta-information.

But ultimately (in the case of mathematics), who cares how long it took you? The coin of the realm is proving quality theorems, not proving those theorems in the shortest possible time. Time spent has absolutely no bearing on the usefulness, or impressiveness, of a proven theorem. It may be useful information to know in some of the natural/social sciences, but in mathematics it is simply not relevant.

1

To avoid cognitive bias.

Unfortunately, humans are just humans. The problem with human psychology is that we must compare, evaluate and decide "what's better". If there are no meaningful and measurable qualities to compare, we compare any qualities that we can get our hands on. There are many mundane examples, e.g. cameras being compared by useless property of megapixels, just because it's a solid, available and easy to compare number while the property we want (quality of pictures) is not easy to measure, quantify and compare. Many of us would not be able to help but to judge research by it's duration where judging its merit would prove too hard.

Length of research would enable superficial comparison of unrelated research and that could lead to bias. Possibly shorter research being treated as "not as serious" as longer ones.

1

Because efforts on most research projects are interspersed with other activities (e.g. other projects, grant-writing, reporting, teaching, grading, serving on committees, attending conferences, reviewing, organizing events, interviewing, advising [even other students, informally] etc.) the time between starting and finishing a research project often seems embarrassingly long when stated without all that other context.

Authors want to avoid readers' sensible reactions of "What? That took how long? I could've sat down and done it in a tenth of that time!"

None of the other context is relevant to the contribution that the work is making to advance the common state of knowledge, so it doesn't need to be included. Further, the amount of time invested in getting that advance in knowledge isn't necessarily correlated to the value the work has in advancing common knowledge, and the latter is what really counts.

Also, a researcher will spend a LOT of time reading a body of literature that informs more than one paper. This means the first paper takes longer start-to-finish than another paper which is the same amount of work beyond the literature review. How would one properly allocate that common background time to multiple papers, especially when they don't know how many there will be in the future at the time of writing any current one? How would you count all the time spent searching for, starting, or reading papers that turned out to be dead ends and not particularly informative?

The backstories sometimes come out in less formal settings, but even then the number of hours/weeks/months/years it took researcher X to do something doesn't necessarily help researcher Y (who has a different set of other tasks going on) determine how long it would take them to do something similar.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.