0

A study compared two groups (differing on factor A) on ability X and found that X differs between the two groups (effect of A). However, the groups also differ on a not interesting variable B (say, "age"). When using B as covariate in the analysis of A on X, there is no effect of A.

In the statistical analysis (ANCOVA) the authors present the statistics for X corrected for B (i.e. "no significant effect of A on X"), however, next to this analysis, they present the effect sizes (Cohen's d) of the raw/uncorrected values of X as support for their hypothesis ("large effect of A on X"). They do not attempt to make it obvious that these effect sizes are based on scores without correcting for B.

I think they have to present the effect sizes of A corrected for B (i.e. after regressing out the effect of B on X). However, they don't want to, possibly because they have a strong hypothesis about A affecting X.

Is what they are doing correct, is it normal, is it misleading or even fraud?

Improve this question
1

1 Answer 1

Reset to default
1

You found out, so apparently there is enough information present in the article to see what they were doing. So that in my mind rules out fraud.

Whether or not it is correct, normal, or misleading depends on the finer details. There is also another possibility: it is simply a mistake. Researchers are human after all.

Improve this answer
2
  • 1
    I found out through communication with the authors, the information is not present in the paper. There I also explained what I though would be correct (i.e. presenting effect sizes based on data corrected for B), upon which they responded that what they are doing is what they want to do, thus excluding human mistake.
    – Mark
    Sep 25, 2017 at 8:36
  • The fact that they want to report this does not exclude human mistake. Statistics is a necessary evil for most substantive researchers, so it is common that they are not as up to date as one would have liked.
    – Maarten Buis
    Sep 25, 2017 at 10:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .