I'm struggling with how to present a research article. The research questions were along the lines of "Does the difference between X and Y predict Z?" It turns out that the difference between A and B predicts Z better than the difference between X and Y. We explore reasons for this finding in the discussion section. Should the literature review (which comes before the research questions) address previous research on the difference between A and B? Can I put some of that literature in the introduction? Or is it disorienting to then come to the research questions and find that they're only about X and Y, not A and B? Should the research on A and B go in the discussion section instead? The field is education, if that matters.
1
-
2Are you forced into a particular outline of the work? Can you have an introduction or abstract in which to mention things of importance? More important, since the research questions focused on X and Y do you have anything more than anecdotal evidence about A and B? If you aren't testing A and B, how do you come to "conclusions" about them? Something seems off.– BuffyJul 28, 2022 at 14:27
Add a comment
|
2 Answers
If you actually set out to study:
"Does the difference between X and Y predict Z?"
...you would never be able to find:
It turns out that the difference between A and B predicts Z better than the difference between X and Y.
There's just no way that this conclusion occurs by accident.
At some point in your project, perhaps your goals changed, but you certainly must have studied which difference, A-B or X-Y, best predicted Z, so that's your actual research question for the study.
If you actually also tested C-D and Q-R and all sorts of other possible predictors, you should report those as well. You should definitely make clear what variables you worked with and what your procedure was for testing them, as your study has very different implications if you just kept testing different things until you found one that was "significant".
To summarize and address the question you asked, your framing of the paper should reflect the actual study you did, not the study you wanted/expected to do. You should contextualize all the variables you measured and tested, regardless of the results. You should make clear what results you might have expected for all of those variables ahead of time based on previous research in the literature, including literature that is contradictory. Your discussion should explain how your results, both positive and negative, affect your field and suggest future work. Your discussion should also make clear possible weaknesses in your work, including the possibility that testing multiple hypotheses raises the likelihood that the relationships you find are false-positives (or, alternatively, if you correct for testing multiple hypotheses, how the reduced power in your study makes it likely you've missed some true positives).
answered Jul 28, 2022 at 16:00
-
1+1 Indeed, the description in the question immediately sets off one's HARKing alert. Jul 28, 2022 at 16:16
-
(Although, following the Wikipedia article that I linked in my previous comment, the notion HARKing might be overly harsh in this case, since the question does not indicate (at least not to me) that OP tries to "[falsely portray] post hoc hypothesizing [...] as a priori hypothesizing".) Jul 28, 2022 at 16:22
Your introduction should preview and motivate your work. Your statement that the literature review comes before the research questions seems like a bad choice in your case, because the previous literature is off to the side of what you studied. It seems to me that a clear way to organize your introduction would be:
It is important to predict Z because [reasons, citations]. Previous researchers have attempted to predict Z using the difference X-Y [citations]. Our main result is that the difference A-B is a better predictor of Z than X-Y is ...
Of course, as other answers have pointed out, you should make should be concerned if this hypothesis was formulated after you analyzed the data. Ideally, you would collect new data on (A,B,X,Y,Z) and see whether the superiority of A-B over X-Y held up in the new data.
If this isn't possible, you should at least analyze how many different pairs (A,B) you could have considered as predictors and do the appropriate statistical analysis to make sure you aren't making the error of this XKCD. The norms regarding how careful you should be here will vary from field to field.
