Are there any acceptable reasons for omitting data in a publication (i.e. "data not shown")?

Question

I was recently going through an article (which I was told is fundamental in the area of my new project) published in a respectable medical journal. In short, the paper is about the similarities between a particular sub-type of tumor cells versus a particular type of stem cells in the body, then they go on to investigate what that similarity might indicate.

Barely two pages in and I realize that the authors omit what I consider to be critical data when motivating the use of two particular proteins as markers in establishing their fundamental assumption, that their immunohistochemistry findings are representative of the reality and that their model is valid:

Accordingly, we screened all known ..... markers against our ..... data to determine which, if any, decrease with differentiation (data not shown). Of all the potential ... markers, we found that X and Y are the best candidates, and they were therefore used in this study.

Now it might not be a big deal to some but I find it important to see that omitted data, since:

1. I am not that informed in that particular type of biology

2. Seeing that their "real" findings are built on the results of the aforementioned screening, the validity of their research is practically depending on the decision of using those two markers.

Which brings me to my question: Are there any cases where not showing "critical" data in such a manner is acceptable or common? I know that some journals do not accept "data not shown", but obviously it does happen in better journals as well...

Answer 1

I think it is perfectly reasonable to not "show" data. For example, consider a multivariate randomized control trial with two groups. If there are a large number of partially correlated dependent variables, the data are not suitable for graphical or tabular presentation. What we are potentially interested in is if there is a difference between the two groups. We can "see" that from a single sentence about a statistical test and a comment that the data are not shown (so people do not go looking to find the figure).

In a biology example, maybe you are counting the number of intact cells after two different treatments. There may be hundreds of slices that result in two numbers. What exactly do you want to see?

Answer 2

Critical data? Probably not. But, I have seen many papers with data left out simply because of paper length restrictions, and in all cases other graphs and data appropriately contributed to the papers' fundamental arguments.

In the cases where I wanted to see the data that was omitted, I've written one of the authors and they have almost always forwarded it along.

Answer 3

What constitutes "data" is a highly subjective issue. If "data" means "all scientific output," in many cases this exceeds what can reasonably be shown in a paper.

In molecular simulations (my particular field), we may generate gigabytes or even terabytes of data for individual papers. This data needs to be crunched down and represented in figures that process the data and make sense of it to the reader, as it is plainly impossible to show the reader the same data over and over again. Thus, we choose to show only the most essential information, rather than deluge the reader with more information than can be handled either visually or in tabular form.

Answer 4

It is of course difficult to judge this particular paper without having seen it so I will stick to general terms.

I do not see any reason for not representing all data in some form. When I say represent I mean that the complete data set can be given in terms of, for example, statistical measures which would at least provide some way to see or judge whether the subset shown is truly representative. But, I still have a hard time seeing a good reason for omitting it without clearly explaining on what grounds.

Journals allow supplementary information and so it seems reasonable to use that option if it exists in th eparticular journal. In very old papers where plots were mae by hand, there may be some excuse if not everything is included.

As I see it one of the fundamental principles of scientific publication is reproducibility. That requires access to all data. There are of course instances where this i snot possible such as when patient journals are involved.

Answer 5

From the way you've described it, it sounds like the choice of markers was somewhat arbitrary. Presumably they had figures that demonstrate that the markers they did choose are good at marking whatever-it-is that needs to be marked. Thus, the fact that they looked at howevermany others is immaterial to the scientific content of the paper and is rightly omitted.

Now, that data could well be useful to someone else for whom markers X and Y are inadequate (maybe because of expression problems or tissue type or whatever). It would be useful to the scientific community to know what that data is (if indeed it was done carefully enough to be worth anything--maybe it wasn't, but it didn't matter because X and Y checked out). But there's no requirement to be especially helpful to the rest of the community (or to avoid being sloppy in some areas as long as you go back and do it carefully/right once you know what you're doing). So it seems reasonable to me.

(Again, inferring from what you've said. If that screen was what told them that X and Y had the properties that they thought, and there is no other confirmation, then they'd better show that data in convincing detail!)