I have written an article about using technique X as a methodology for lecturing an Information Technology classroom. At the beginning of the experiment I hypothesize that the use of this technique would have a negative effect on the learning outcomes of the students, even though a colleague was pretty sure that it was going to be the opposite way around. Anyway, when the experimentation part ended I was right, the methodology applied was worse than the old method applied and we wrote a paper about this experience. The problem is that we tested the methodology only in a small group of students, 30 approximately, but we have some doubts if it could get rejected because the sample is too small considering other experiments in which there are involved hundred of students. Bottomline, how to deal this issue in a research paper? Should I only present the results and not make any statistical hypothesis testing? I have performed an h-test, but one colleague told me that still the sample is too small. Any help?
The proper way is to conduct a power analysis and present its results in the manuscript. Of course, such an analysis
might show that your sample size is too small for conclusive results
should have been conducted prior to the study.
As a result the paper might be rejected during review, but one could argue that it should be rejected. If the power analysis shows that you need to increase sample size, you could repeat/extent the experiment until you achieve sufficient statistical power.