Research has shown that most of the published studies in various fields are false. For example, the Reproducibility Project: Psychology only managed to confirm 36% of previous 100 psychology studies had a statistically signifiant result the second time. Also, only 6 of 53 studies considered landmark in cancer were successfully replicated. In these circumstances, when such a quantity of false studies pass the peer-review process in most scientific fields, how can we filter authentic information from fields we don't have expertise in?
First thing to consider is that research can never provide absolute certainty. Any result you find is only preliminary. You can have various degrees of preliminary: a single study with an acceptable but not brilliant design can provide interesting clues or first steps for a larger project, but not much more. It is right and productive that these studies are published, but don't treat them as truth.
Second thing to consider is that the problem is not necessarily with the articles themselves, but with the lack of replication. So, if you find a result that interest you, try to find replications. If none exist, then you need to be more careful.
The p-curve (see p-curve.com) has been proposed as a way of identifying whether a set of results is likely to have been p-hacked. The term and technique was coined and/or developed by Uri Simonsohn and colleagues, who blog about replication and p-curving at Data Colada.
- "P-Curve Won’t Do Your Laundry, But Will Identify Replicable Findings"
- "Ambitious P-Hacking and P-Curve 4.0"
- "P-curve vs. Excessive Significance Tests"
The technique has been applied by several research groups besides Simonsohn et al, ie Head et al 2015
There have been critics of p-curves (including John "most published finding are wrong" Ionnadis ), as well as other techniques such as excessive significance tests and the distribution of published p-values.
- Simonsohn, Simmons, Nelson (2014) "P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results" Perspectives on Psychological Science V9(6) p.666-681
- Simonsohn, Nelson, Simmons, (2014) "P-curve: A Key to the File Drawer," Journal of Experimental Psychology: General, V143(2), p.534-547
- Simonsohn, Simmons, Nelson (2015) "Better p-curves" Journal of Experimental Psychology: General.
- Ioannidis and Bruns. 2016. p-Curve and p-Hacking in Observational Research. PLoS ONE.
- Head et al 2015 The Extent and Consequences of P-Hacking in Science. PLoS ONE