I was reading through an article about research design. The authors gave an example on the relation between internal and external validity which I thought might be an important area of consideration when designing research proposals.

"A restricted study population (and exclusion criteria) may limit bias and increase the internal validity of the study; however, this approach will limit external validity of the study and, thus, the generalizability of the findings to the practical clinical setting. Conversely, a broadly defined study population and inclusion criteria may be representative of practical clinical practice but may increase bias and reduce the internal validity of the study."

Broadly speaking, what questions can be asked to help decide one should strike a balance between the internal and external validity in our research proposal designs? Is it necessary then to carry out a pilot study first to establish some level of internal validity before moving on to establish external validity in a much bigger and formal follow-up study?

Crosspost: Quora

  • Good question (+1). I think that you would have better chances of getting good answers to such focused research methodology question on Cross Validated (you might have some luck on Quora as well, so crossposting was a good idea). – Aleksandr Blekh Dec 2 '15 at 6:41
  • For anyone tracking, it ended up here – mflo-ByeSE May 5 '17 at 19:46