There has been quite a bit of literature on this topic. Because your question is broad (regarding acceptability, security, reliability, and validity), I have provided a brief overview; these articles should provide you with a starting point to locate additional information.
Reliability and Validity
A recent article providing an overview of the validity of different student evaluation methods in general may be of interest; the article does address online administration as one component impacting validity (Spooren, Brockx, & Mortelmans, 2013).
In a longitudinal study of over 63,000 student evaluations over 10 years (comparing paper in-class with electronic out-of-class evaluations), no significant differences in evaluation scores were found due to administration method (Risquez, Vaughan, & Murphy, 2015). However, one study had nearly 200 business management students evaluate the same teaching with both on paper and electronically within two weeks (Gamliel & Davidovitz, 2005). Although the mean scores were similar between the two groups, stability of scores was higher in the paper method. Thus, if the same teacher were to be evaluated multiple times, there might be more variance if the method of administration changed. The authors suggest that paper evaluations may allow students to look at the written horizontal profile of their responses, which they may recall on the second administration. In contrast, electronic systems that display questions differently may remove this effect.
A small study of over 120 medical students found that those who submitted paper evaluations immediately after training gave more favorable reviews than those that submitted electronic feedback at a later point in time (Palmer, 2009). The author posits that it is possible the rapport between the instructor and students immediately after session lead to better reviews, while those providing electronic evaluations did not have that social pressure. However, they state it is also possible that the time required to log-in and take the electronic survey may have been irritating, resulting in more negative evaluations.
Response Rates and Timing
Adams and Umbach (2012), reviewed 135,000 online course evaluations completed by 22,000 students. They found that students were more likely to respond to evaluations for courses within their major, and less likely to respond for ungraded courses or if the student was receiving a D or F grade in the class. They also noted some differences in response rates across majors, and evidence of survey fatigue that may impact response rates. Similarly, a study of graduate students using online course evaluations found that those students who responded earlier had more positive course ratings and expected to get higher grades, while those who submitted information later had lower grade expectations and course satisfaction (Estelami, 2014). The author suggests that universities must be aware that early/late responders differ in several specific ways that may impact the data they provide. In contrast, Pegden and Tucker (2012) found that the timing of completing the evaluation electronically did not impact responses, as evaluations were generally stable regardless of when there were completed.
Another article based on online evaluations at the University of British Columbia provides guidance on how to assess adequate response rates to enhance reliability and validity (Zumrawi, Bates, Schroeder, 2014). Nulty (2008) provides suggestions for improving response rates for teaching surveys, and compares paper/electronic methods; he suggests a mixed method approach may result in the highest response rate.
A study of nearly 1,000 students found fewer students completed the electronic version compared to a paper survey; students reported frustration with the site’s complicated and time consuming log-in process and were concerned about anonymity (Dommeyer, Baum, & Hanna, 2002). However, those authors caution that these burdens may be reduced as technology improves. Indeed, a more recent study of over 1,000 students found that students preferred the electronic evaluation option because they felt it was more anonymous, convenient, and presented less time pressure than completing the survey in class (Kinash, Knight, & Hives, 2011). A study of student perceptions about what is important in an online evaluation system reported a list of student concerns related to anonymity, ease of use, and being made aware of the impact of their responses (Nevo, McCLean, Nevo, 2010); this study provides information on a range of student concerns related to the evaluation system, with suggestions for other universities to follow.
Anecdotally, my graduate school recently transitioned from paper to online evaluations. I prefer the online evaluations because they allow me time to gather my thoughts, and I can provide more structured feedback regarding the class. Log-in is simple and is based on the universal log-in for our university; I take it on faith it is de-identified before being sent the instructor. Given how small some of our graduate courses are, that actually feels more anonymous than turning in a paper survey with my handwriting. We receive multiple email reminders to take the surveys during the last two weeks of class, which indicate the overall percent of students that have already responded (a form of social pressure, I suppose). There are only two things I dislike about the system. First, the response period ends before the final exam period begins, so you cannot include thoughts about the exam in your evaluation (the survey ceases to function). Second, responses are only available to the department chair, tenure committees, and the professor being evaluated. At my undergraduate institution all course evaluations were public, so students could read the evaluations of past classes to determine whether and when to take the course.
- Adams, M. J., & Umbach, P. D. (2012). Nonresponse and online student
evaluations of teaching: Understanding the influence of salience,
fatigue, and academic environments. Research in Higher Education,
- Dommeyer, C. J., Baum, P., & Hanna, R. W. (2002). College students'
attitudes toward methods of collecting teaching evaluations: In-class
versus on-line. Journal of Education for Business, 78(1), 11-15.
- Estelami, H. (2014). The Effects of Survey Timing on Student
Evaluation of Teaching Measures Obtained Using Online Surveys.
Journal of Marketing Education, DOI: 10.1177/0273475314552324.
- Gamliel, E., & Davidovitz, L. (2005). Online versus traditional
teaching evaluation: Mode can matter. Assessment & Evaluation in
Higher Education, 30(6), 581-592.
- Kinash, S., Knight, D., & Hives, L. (2011). Student perspective on
electronic evaluation of teaching. Studies in learning, evaluation,
innovation and development, 8(1), 86-97.
- Nevo, D., McClean, R., & Nevo, S. (2010). Harnessing information
technology to improve the process of students' evaluations of
teaching: An exploration of students' critical success factors of
online evaluations. Journal of Information Systems Education, 21(1),
- Nulty, D. D. (2008). The adequacy of response rates to online and
paper surveys: what can be done?. Assessment & Evaluation in Higher
Education, 33(3), 301-314.
- Palmer, R. (2009). Evaluation of teaching--paper versus electronic
methods. Medical teacher, 31(12), 1094-1094.
- Pegden, J. A., & Tucker, B. (2012). Does the timing of evaluations
matter? An investigation into online student feedback and whether
timing has an impact. Studies in Learning, Evaluation, Innovation &
Development, 9(1), 55-65.
- Risquez, A., Vaughan, E., & Murphy, M. (2015). Online student
evaluations of teaching: what are we sacrificing for the affordances
of technology?. Assessment & Evaluation in Higher Education, 40(1),
- Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the Validity of
Student Evaluation of Teaching The State of the Art. Review of
Educational Research, 83(4), 598-642.
- Zumrawi, A. A., Bates, S. P., & Schroeder, M. (2014). What response
rates are needed to make reliable inferences from student evaluations
of teaching?. Educational Research and Evaluation, 20(7-8), 557-563.