Is changing research methods/ methodologies in the middle of running a research project a good thing? Are there any best practices for doing so? Links to projects that document a methodological shift mid-project would be much appreciated.

  • 4
    Could you provide a few more details regarding the project you are talking about, as I think the answer would vary a lot depending on that? – TCSGrad Feb 22 '12 at 11:07
  • +1 for adding more detail as to exactly what you're looking for. – Fomite Feb 22 '12 at 12:51
  • It is a project in social sciences and the methods/ methodology appears to be moving between case study research - action research - evaluations. I arrived mid-project and trying to make sense of it. – user1871 Feb 23 '12 at 9:56

Do you keep a methodology notebook? If not, it is high time you start now.

It is important in the experimental sciences to always document exactly how the experiment is performed, and it is often recommended to keep a methodology notebook aside from your data notebook. All current and past/obsolete methodologies should be documented and dated (so it is clear exactly when your lab switched to a new methodology). All data on experiments should include reference to the correct methodology. (A lab wiki may be a good way to manage this.) For a bit more discussion you can see this guideline from the neurosurgery department of UCSF.

Changes in methodology may or may not be a good thing. A change of methodology obviously introduces a new variable into your research. And depending on the research outcome, the data pre and post methodology change may or may not be compatible, and it may or may not be ethical to present the data as one single dataset. So if your experiment only has limited number of runs for which systematic bias that may be introduced by the new methodology cannot be statistically ruled out, you may be better off not switching. (Unless, of course, you are in the biomedical sciences where sometimes methodological shifts are compulsory, even though it means essentially aborting the experiment.)

Methodological shifts are not forbidden per se, as long as all the methodology are clearly documented. The issue is more with how you interpret the data afterwards: it maybe necessary to present your data as two separate experiments instead of one single experiment. (Depending on your field there are different culturally accepted level of "fudging" when it comes to the methodology section of a paper; you should always have an argument handy as to why changing the methodology does not really change the experiment.) As long as each data point can be clearly referenced to the method that was used to obtain that data point, your experiment is reproducible and ethically you've covered your behind.

(This, of course, does not address the bureaucratic issue of whether your funding agency and such would frown on deviating from the funding proposal, and whether your research institution have specific guidelines concerning methodology shifts.)

  • Although my field is social sciences I found this very insightful, thank you. I especially take on board the one about clear documentation. – user1871 Feb 23 '12 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.