Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

On Skeptics Stack Exchange I recently asked a question where I specifically asked for a study backed by a university, and someone edited it, changing it to the phrase "peer reviewed study." If I say "peer reviewed study" does that describe a study that is accurate, reliable, unbiased, etc? Is there any better description for the type of study I'm looking for?

share|improve this question
11  
Incidentally, one objection to "a study backed by a university" is that it's not an accurate description of how universities work. Universities don't back or endorse studies, even if those studies were carried out by their employees. (In fact, sometimes different faculty members at the same university publish conflicting studies.) News reports often give the wrong impression about this, by saying "a Harvard University study" as if Harvard were the primary actor here, but in fact the researchers are responsible for the study, while Harvard as an organization has no official opinion about it. – Anonymous Mathematician Feb 20 at 7:05
1  
...and depending on the precise field, a substantial amount of research may be done by non-university institutions – Andrew Feb 20 at 7:45
4  
To quickly illuminate that last comment - in yesterday's Nature, 530(7590), there are 16 research articles. One has an entirely non-university author list (all are at MRC Laboratory of Molecular Biology). Eight others have one or more non-university coauthors, who come from places ranging from research institutes to observatories, including at least two private companies. It's a diverse world of research out there! – Andrew Feb 20 at 10:55
4  
One could even argue that papers whose authors are in a research lab have a slightly higher probability of being accurate than those from a university, since it is less likely that a crucial part of the work has been conducted by a hasty student that is in a hurry to finish his/her thesis. – Federico Poloni Feb 20 at 13:27
1  
@FedericoPoloni: my non-university institute has lots of students that are in much hurry to finish their thesis before money stops... – cbeleites Feb 20 at 22:12
up vote 13 down vote accepted

Determining whether a study is accurate, reliable, unbiased, etc. can actually be quite difficult, particularly for tricky and difficult to measure phenomena. This is what scientists argue about amongst themselves, and the consensus that emerges is best interpreted as the current best understanding of a subject.

Peer-reviewed publications are a good starting point, because the peer-review process strives to ensure that they are in fact accurate, reliable, unbiased, etc. The publications are written and reviewed by humans, however, who are not very good at those things, and as such some bad articles get published and some good ones don't. Likewise, there are many different levels of quality in peer-reviewed journals. Still, for all its flaws, peer review is a lot better than not-peer-reviewed at filtering out things are are inaccurate, unreliable, badly biased, etc, and thus "peer-reviewed publication" is typically a good heuristic for "probably worth considering as a reasonable source of information."

That is not, however, the end of the story on whether an article is a good source of information. A good next step is to see what other scientists say who reference the article in their own publications. One article does not an established fact make; many in agreement do.

share|improve this answer
5  
A good next step is to see what other scientists say who reference the article in their own publications. In particular, if other scientists have reproduced the result then one can be much more confident that it's right. Medicine has a real problem because there is a pattern where people keep trying things until they get a positive result, then they publish the positive result, even if it's a statistical fluke. If other people try to reproduce the result and fail, it tends not to get published, because journals don't want to publish negative results. – Ben Crowell Feb 20 at 14:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.