Research has shown that most of the published studies in various fields are false. For example, the Reproducibility Project: Psychology only managed to confirm 36% of previous 100 psychology studies had a statistically signifiant result the second time. Also, only 6 of 53 studies considered landmark in cancer were successfully replicated. In these circumstances, when such a quantity of false studies pass the peer-review process in most scientific fields, how can we filter authentic information from fields we don't have expertise in?

  • 2
    Welcome to Academia SE and thank you for your question. Can you please edit it to clarify whether you want to ask about p hacking (as indicated by your title) or general problems leading to irreproducibility (as described in the body of your question). I recommend the former, as the latter would be rather broad.
    – Wrzlprmft
    Dec 27, 2016 at 16:40
  • 5
    From (RP confirmed 36% of results) does not follow that (64% of results are false). Dec 27, 2016 at 19:48

2 Answers 2


First thing to consider is that research can never provide absolute certainty. Any result you find is only preliminary. You can have various degrees of preliminary: a single study with an acceptable but not brilliant design can provide interesting clues or first steps for a larger project, but not much more. It is right and productive that these studies are published, but don't treat them as truth.

Second thing to consider is that the problem is not necessarily with the articles themselves, but with the lack of replication. So, if you find a result that interest you, try to find replications. If none exist, then you need to be more careful.

  • Your first point is very important. p = 0.05 always means there is a recognized possibility that the result will not replicate even if the research was done and analyzed as perfectly as possible.. However, the OP's invocation of "p-hacking" indicates that they are concerned with papers whose findings are not robust to replication b/c their p-values are not accurate due to design or analyses that compromise a valid interpretation of the published p-value..
    – N Brouwer
    Dec 28, 2016 at 18:31

The p-curve (see p-curve.com) has been proposed as a way of identifying whether a set of results is likely to have been p-hacked. The term and technique was coined and/or developed by Uri Simonsohn and colleagues, who blog about replication and p-curving at Data Colada.

The technique has been applied by several research groups besides Simonsohn et al, ie Head et al 2015

There have been critics of p-curves (including John "most published finding are wrong" Ionnadis ), as well as other techniques such as excessive significance tests and the distribution of published p-values.

Some references:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .