In our university, we have a discussion about different question types to use in a module in our Electronic Learning Environment called 'self-tests'. We use this module to stimulate active processing of learning material by students in an automated fashion, enabling us to offer courses with a high degree of interactivity while remaining feasible for large numbers of students.
Our discussion concerns specifically which question types we need to implement in the Electronic Learning Environment. Some example of question types are the following:
- Multiple choice questions
- Open questions
- Ranking questions
- Images with 'hotspots'
- 'Likert'-type scales and matrix questions ('arrays' in LimeSurvey)
- Sentences (or complete paragraphs) where words have to be entered to complete the sentences
Some applied examples would be:
- Please organise these concepts to reflect the developmental stages distinguished by Piaget.
- Which of these types of sampling are appropriate? In this painting, point out where the artists uses [INSERT BRUSH TECHNIQUE HERE - sorry, I'm a psychologist, not an art scientist :-)]
- Given the above material properties, how much load can the bridge bear?
A colleague recently voiced the position that all other question types 'boil down to yes-no decisions' and that therefore, there is practically no variation in the types of information processing students engage in when working with the different question types.
However, in this discussion, nobody has as yet managed to produce any meta-analyses of the evidence, or in fact other forms of empirical, or for that matter, theoretical, evidence. At the same time, I can't imagine this hasn't been studied yet. However, queries in Google Scholar using keywords such as 'information processing', 'cognition', 'formative tests', 'interaction' and 'learning' didn't yield anything. Lots of opinions and examples, but no evidence.
So my question is: does anybody know of literature presenting empirical or theoretical evidence regarding differences in processing between different question types (methods of interaction)?
This is the kind of thing academics (as teachers) should all have readily available I guess - but I'm afraid I don't, and I don't know anybody who would.
I don't have any 'dogs in this fight' - if there is evidence that all question types perform equally, that's fine. I just want evidence, regardless of which way it points :-) As long as the evidence is methodologically and statistically rigorous I'm happy.
One answer would be, of course, "it's not about the question type, it's about what exactly you do with it and how you embed it in the course". However, this ignores the fact that different question types enable/prohibit different problem solving approaches. It would basically just change the question into "assuming the question types are applied properly, which serve which goals best?"
I'd be very grateful for any pointers towards literature where (preferably experimental) evidence on this matter is presented.
[this question is kind of like a more in-depth version of this one, which is about whether online quizzes can work in general]