18

Given the recent shutdowns of many universities across the world, there's an increased interest in online education techniques.

In my case, I am a TA for a quantum chemistry class, and I'm finding it daunting to transition to online teaching. At first, I got really excited because I thought of taking advantage of this situation to run a flipped discussion section: I'd record lectures in advance, and then have virtual office hours where students could Zoom/Skype/Google hangout with me and we discuss in the same chatroom (so everyone hears the conversation).

However, in what I suspect to be an attempt to distance themselves from something nice and free/far cheaper (like edX) which has been doing this online routine for a long time, my university has required that we livestream our lectures/discussion sections. This is purportedly to be in the aim to make their online experience as college-like as possible, but it's problematic for multiple reasons. First, lectures are at 9AM EST... Do we expect students to tune in at 10 PM in Japan? Or 3 AM in Hawaii?

Second, we should use the silver linings present in this university shutdown: video lectures = students get to go back and re-listen to a part that confused them! From my perspective, this university requirement goes against an attempt to maintain normalcy: normalcy should not be defined as continuation of the same daily routine (impossible for many), but rather a continuation of helpful instruction.

I intend to find out whom to email about this, but in the meantime, what resources/advice would you suggest for an instructor who suddenly has to transition to the digital classroom?

10

This is marked for community ownership so constructive edits will be welcomed. It is, however, more directed at those who can design a course, rather than a TA who has limited control.

Background

Many of the teaching techniques we use are not optimal for learning. They have been designed for economic efficiency since they scale. Some of them have been a bit obsolete since the creation of the internet. But face to face classes in real time are no longer necessary. Lecture is efficient (one teacher, many students, synchronous), but again, not optimal. Testing is a poor way to guarantee competence. This current emergency can be considered to be an opportunity to rethink everything we do toward finding better ways to educate people. We are forced to act and the old ways don't work in a new environment. We should be looking for new ways.

The essentials of teaching are Delivery, Practice, Feedback, Assessment. All of these require Two Way Communication. They don't require synchronicity or exams. The first job is to teach people, not to grade them. If you get that part backwards then you are only "teaching" people who don't need to be taught.

Every student is different. Nearly every student learns differently from the way their instructor does. The instructor has been proven to stand out in a particular system in which others have failed. As an instructor you have an obligation to every student willing to put in the effort, not just to the ones naturally predisposed to "win" in some system. Not every student starts at the same place with the same understandings. The same words don't necessarily have the same meanings to different people.

Some people have handicaps that prevent certain kinds of interactions. Hearing impairment and vision impairment are obvious ones, but there are others.

Learning is individual, but the process need not require isolation. It can be a communal process with learners helping each other. Denying students the option of working together is sub-optimal. A course can be a communal activity, not a hierarchical one.

Some students will succeed no matter the system. They don't really need you for anything more than pointing to important sources/ideas. But most actually require your assistance since insight into a topic can be hard to come by. Many students actually need to be taught how to learn. If you change the system of learning they may need to be retaught how to internalize ideas and turn them into competencies. It isn't automatic or obvious.

Learning isn't "consuming" material. It requires active participation. It requires a change in the brain (The Art of Changing the Brain by James E Zull)

All of these things have long been taken into account in designing educational systems, but the current practice is a compromise that values economic (scale) values over educational ones. Not everywhere, of course, but the exceptions seem few to me.

Let's assume that everything is on the table, except that students and instructors can no longer meet face to face in groups. Meetings of any kind need to be small and optional with alternatives provided. Computer communication of some kind is needed, but costs are an issue as is the privacy of all participants.

What to do, what to do.

Ideas for Delivery

There are a lot of ways for students to have access to the material of a course. Books are traditional. Lecture has replaced most other methods. But lectures are longer than they need to be. Some research I read suggests that lectures of 10 to 15 minutes are more efficient than hour or longer lectures (https://spokenimpact.com/what-is-the-ideal-length-for-a-presentation/). Those contemplating video should take note of that. Video also needs to be accompanied with a transcript to aid those with impairments.

Moreover, students need to be able to review material. Video is difficult in this regard since it is hard to quickly return to a small part that needs review without watching longer sections, which leads to boredom. Transcripts can mark the time that important ideas are introduced.

There is a fine line between having good and poor production values in video. If they are too good, a video talk becomes just entertainment with poor learning, unless other means are used to encourage/enforce engagement. If they are too bad, then students will get lost and frustrated. Video professionals are good at this sort of thing, but it is hard to bootstrap immediately at scale.

Delivery alone is just media. Students need to incorporate what they see in a lecture, long or short, into their long term memory and into their effective competencies. More is involved than the eyes and ears. Students often think that the will automatically "learn" what is presented to them in a lecture, talk, demonstration, etc. but it is anything but automatic.

Caveats: Any "solution" that requires real-time synchronous presence on the internet will probably fail. And it will fail invisibly for some. Connections are dropped, servers overload, etc. It is hard enough to keep students off their phone when in a class. It is probably impossible to do in a real-time internet lecture. Other distractions will intrude. Moreover, unless you have an exceptional communication system, students won't be able to interrupt you when you make a mistake, as you will. No questions are likely to be asked in real time. It is also difficult for a presenter to give a lively talk with no audience feedback at all. Especially if you are trying to do this under emergency conditions without training and experience.

But if you absolutely must use synchronous delivery you need to do two additional things. The first is that you need to record and post your lectures so that those who must miss the original don't lose access.

The second, just as vital, is that you provide a real time feedback to yourself (or to an assistant sitting by your side) so that questions can be asked as you lecture and so that students have a way to mention/note errors you make.

And make sure that students know what to do in the case that you lose the server and the ability to broadcast. Avoid chaos and uncertainty as much as possible.

Ideas for Practice

Student practice exercises and tasks may not need to change much. Students, depending on the field, need to write, solve problems, and other things as appropriate. For the most part this can probably proceed largely unchanged. Instructors have access to many such student exercises. If the students have a text book they probably have a bank of suitable exercises and questions.

And note that practice and reinforcement is probably the most important element in a delivered education. The "content" that a student needs is, for the most part, available to them. There are exceptions for very advanced work, but the world's knowledge is pretty much available. Instructors help students choose the important things and avoid the garbage, of course, but it is up to the students themselves to learn by reinforcement (combined with feedback). Whatever the topic, competence comes from repetition and reinforcement. The only thing I teach nowadays is Tai Chi. We have a saying that you need to repeat a movement ten thousand times to learn it. Somewhere in those ten thousand tries, insight will start to develop. The same is true in things like mathematics and writing poetry.

So, remember that the things that you make the students do to learn are much more important than the things you say. Good course design focuses on the student tasks, not the lectures.

I caution, however, that copyright considerations may apply if an instructor tries to utilize exercises from copyrighted material. They are, in a sense, publishing things they send to students in an online course. Caution and legal advice is useful.

Ideas for Feedback

Feedback is two way. You need feedback from the students on your presentations and assignments. Students need feedback on their own work. A number or letter grade on a paper isn't feedback. It gives no basis for understanding how to improve.

If the materials presented are well vetted, such as textbooks, the need for feedback to the instructor is a bit lessened. But videos, especially those prepared in the moment are likely to contain errors. Shorter videos are easier to fix than longer ones. But, unlike face to face classes, the instructor isn't likely to be interrupted when there is a misstatement or an omission. Moreover, you need some indication that the students have actually understood what you have been saying and presenting. Again, in fact to face classes you can look at expressions on their faces for some important feedback.

Lets assume for the moment that you have already taught your students the importance of taking notes during presentations. At the end of a lecture you can ask individual students, perhaps volunteers, to tell you an important idea from the talk. You can do this two or three times and give them feedback on their replies. This can be transferred to online learning. If readings or video presentations are short (and, I assume asynchronous) you can ask students to send you a summary of the lesson. Restrict them to one or two sentences. Another possible request is to ask them to send you a possible exam question on that material. At some point, all student replies, perhaps anonymized, can be made available to everyone. By glancing through the replies you learn whether they are getting the main points and they are also getting the reinforcement they need.

Giving feedback to students on their work is critical to learning. This involves answering questions that they have, but also showing them where they are making mistakes, and why, in their own work. The previous section talked about repetition and reinforcement. It is possible for students to learn and reinforce the wrong message. If you get an incorrect idea deeply ingrained in your mind it can be difficult to dislodge. All student work needs to be commented on, not just graded. Publishing solutions to exercises after the fact can help, but it isn't the same thing.

This idea of individual feedback is one of the most difficult things to arrange in the modern educational system, simply because of scale. My university teaching (and student life) mostly involved classes of fewer than 30 students. At that scale, I could easily give good individual feedback on weekly assignments. At 40 students I found I needed help. Massive courses have a problem here that requires tools - probably better tools than are now available. I'll note that Harvard's CS50 course is delivered to about 800 students at a time. But the staff for the course is about 40 individuals, so the ratio is about 20:1. Some of the staff are graders who comment on student work. Others are TAs who hold individual "recitation" sections of 20-30 students so that questions can be answered and misconceptions dispelled. I worry that moving such recitation online might not work as well at scale unless there are communication tools that let people learn from the questions of others. Many students are reluctant to ask questions. The brave students willing to do so actually contribute to the education of their peers. But, answering a question for a single student, say in an office, is much less productive than answering it in public, so that all can share the response. Tools are needed for this, though they can be pretty simple. University managed email lists subscribed to by students and staff are pretty effective.

But if students produce "papers" of any kind, those need to be annotated and returned. More than a numeric or letter grade is needed. Something as simple as pointing a student to a place they need to review is helpful. But the goal is to foster insight. PDF files can be annotated and returned, but the process is awkward at best. And, again, there is the scale problem. It takes time to review and time to annotate. Then, the student will likely have questions that need to be addressed based on the annotations. The increase in awkwardness using current tools may require more staffing to make things work effectively. Even a simple email to a student takes more time than jotting a comment on a paper and then returning it at the next face session.

Ideas for Assessment

In my view, evaluation and assessment are the most difficult issues we face in moving to an internet model. However, they also, again in my view, have the most potential for long term improvements in student learning, provided that we take the opportunity to learn how to do a better job of it.

The first principle of assessment ought to be that any sort of evaluation have an actual educational purpose, not just a "sorting" purpose. I think that many of us aren't doing a stellar job of this, having opted for easy solutions that scale, rather than solutions that aid student learning. Multiple choice tests are overused and while it is possible to design good ones, it is devilishly difficult. Things that rely on machine grading in general are too likely to also rely on only shallow learning and memorization, rather than deep learning and mastery. If the students can be held in a room and closely monitored this sort of thing tells you something about them, but it can likely be better. Move them outside your purview and it becomes nearly impossible. And it even fails at scale.

While you can do some technological things to lock up a student's personal computer they are intrusive but also ineffective. You can enlist a vast army of TAs to monitor student webcams (intrusive, again) but even there, what happens slightly off screen is invisible. Without such intrusive measures students have access to helpers both human and electronic. Closed-book, closed notes exams are probably a mistake in any case, but online they are difficult to impossible to monitor. There are too many inducements for students to break the rules to really assure validity. Honor codes, such as those of Dartmouth College do have a positive effect, but part of that effect is that the codes have been in place for half a century.

I have some personal suggestions about alternatives to examinations and, especially, high stakes examinations such as finals that account for a large fraction of a students grade. I don't claim these are the only solutions, but am more interested in sparking thoughts about how we can do a better - more learning focused - job of evaluation and assessment. I implemented these more and more as my career progressed and I learned more about students and about learning.

The first idea is to reduce the overall effect of any given exam or test. If you have forty small items you can get at least as good a picture of a student's progress as if you have only one or two large ones. Moreover, high stakes testing too often leads to poor student studying habits, such as cramming for exams and cheating.

One technique I used toward the end was "cumulative grading". A course had 1000 points assigned to it and different student activities were "worth" a certain amount of points. A project might be 100 points. The result was evaluated (and commented on) and a certain number of points was awarded, perhaps 80. I also published the minimum number of points necessary for each letter grade, say 800 for a B. When a student accumulates enough points to "earn" the grade they desire they no longer need to do more. There was no risk. Another feature of it was that if the student wasn't satisfied with the points earned they could re-do the work and possibly earn additional points. Some limits were put in place, but they were generous. The philosophy behind it is "You aren't in this course to prove to me that you don't need to be here." But note that I could do this since the ratio of staff to students was good. This is hard to scale without help.

The second main technique I used and can recommend is to make almost all assessment and evaluation based on student writings and projects, not examinations. And, if the nature of the course permits, use group projects primarily. For my courses the group size was usually between two and five. These groups can meet in cyberspace or, if location and other factors permit, they can meet face to face. The projects might be large or small. They can be graded once or often. Points can accumulate. It is the project that is graded, not the individuals, so everyone earns the same grade unless that is impossible. Peer evaluation is used to award bonus points or to solve issues of non-participation, etc.

If there are a number of group projects students can move between groups if needed. But students also need to be taught how to run an effective group project. In the courses I taught, dividing up the work "equally" was always discouraged and if done, usually leads to poor outcomes. Students need to form teams, not just amalgamations of individuals. In CS this skill will be valued later in their careers.

Also, peer evaluation has to be properly managed. Students don't "grade" one another. They comment on the contributions of a few or all of their team mates. "Who was the most helpful team member? (or three top members) Why? What did they contribute? What was your own chief contribution?"

At a certain scale or with sufficient help you can easily determine who falls into which important grading groups. Who is excellent in all respects? Who shows promise? Who is getting by? Who is deficient? Who has failed? Elaborate point systems in which an 85 is somehow significantly different from an 87 have little real meaning in my opinion.

To emphasize a point above. In many fields, even very technical ones, writing is an important skill. Writing essays on a given topic is good professional preparation for a budding mathematician. Exams can be replaced with such essays, lessening the needs for security and such. They seem to me to be much more value long term than skill at exams. But you can probably expect that students will hate it initially.

The above, of course, are just ideas. But I found them to be more aligned with the teaching mission than the more traditional lecture and test.

Ideas for Communication

To handle all of the above with online courses the main need is for effective communication between faculty and students and, I found, between students themselves. One goal is that what any student sees can be seen by any other student.

Communication tools need not be complex or comprehensive. But they should permit asynchronous communication and should permit many-way communication. Threaded email lists are very useful and don't require extensive facilities. They can be private lists, managed by the university alleviating some privacy issues, open to only class members and staff. Documents can be distributed. Questions can be asked. Questions can be answered at any time of the day or night - even by other students. Someone needs to monitor these to make sure that misinformation is quickly corrected and that any rules about posting or not posting certain content are maintained.

If project groups are long lived, then each group can have its own mailing list and can also use other tools they find for communication.

Another useful tool is a group chatroom. Group chats can be held periodically, though the email lists were more useful for discussing course material. These are necessarily synchronous and so not every student might be able to "attend". But it is easy to capture the complete conversation as a text file and send it to everyone or post it on a course web site.


My own expertise in this comes from helping to design and deliver a graduate program in which students met face to face only once a month for half a day per course. All work was done in groups (of about 5). Face time was only used for things that could not be done on line. We used simple tools for communication. All student-professor communication (other than personal things) was open to all students. Students could ask questions at any time and other students were encouraged to answer from their own expertise. The course was always live, every hour, every day. Questions got answered quickly, if not always perfectly. Students didn't get stuck. The goal was to build a learning community. That community has persisted past graduation.


Minor updates only anticipated.


An interesting alternative view can be found here: https://anygoodthing.com/2020/03/12/please-do-a-bad-job-of-putting-your-courses-online/. Thanks to user curious for pointing this out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.