0

I would like to do a study with a class in which the students are split into two groups, A and B. They will each receive a tutorial on the same topic, but group A will get a different tutorial than group B. I will then give a post-assessment and a week after post-assessment, to explore retention rates among other things. The goal is to compare the methods for tutorial delivery. Specifically, group A will get a tutorial that is modeled on the traditional "expert solving problems at the board" while group B will get a tutorial that requires the students to do self-directed problem solving, with scaffolding.

My concern is that I won't be able to distinguish between expert solved problems vs. student solved problems. Rather, I will be able to distinguish between "when an expert solves THESE problems in THIS topic" compared to when students do so.

In short, I'm worried that my study results will only be valid for a very specific set of circumstances, not generally applicable even to other institutions that teach a similar but not identical course. What are some strategies, if any, I can use to avoid overly specific results?

| improve this question | |
  • 1
    You should be worried that your study lacks power and rigor and isn't valid even for your specific circumstances. Presumably your school has a statistician - find them and make friends with them. – user101106 Jan 29 '19 at 15:08
  • 1
    I think this is not just a question of statistics, but also of research methodology. I agree with CJ59's suggestion, but I would add that your school probably has a department that studies education/pedagogy: find them and make friends with them. This sort of question is unlikely to be a truly novel one, and there could be a lot of existing scholarly work in the field for you to lean on. Also, make sure you're complying with your institutions rules for conduct of research with human subjects and students more specifically. – Bryan Krause Jan 29 '19 at 15:51
2

No study in only a single class will be able to definitively answer such a question. The answer can only come from either a very large study or an accumulation of results over several small similar studies.

If it is an attempt to get at the truth it will tell you very little. If it is an attempt to learn/teach something about experimentation it can have value.

In the US, you would need to run this by your IRB. I don't know if Canada has similar regulations. But you are treating two parts of a class (human beings) differently and need to assure that neither group is disadvantaged in any way. It is unethical to do certain things when you strongly suspect that they are not effective, though I don't think that is a particular issue given your description.

On the other hand, there is already a lot of research on learning that addresses just this question. If you haven't already explored it, you really should.

| improve this answer | |
  • yes, we will need to get research ethics approval no worries :-). I've seen some interesting literature, but it appears that there are three camps that are at odds with each other. I am interested in studying under which condition each conclusion can be made. As is always the case, it is impossible to explain more in a 500 character comment :-P. Good answer, though, thank you. – Michael Stachowsky Jan 29 '19 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.