I am a college student who is currently enrolled in a class where the professor posts lecture videos online with ads enabled. I was wondering if this is legal; the professor is being paid by the university to lecture, but is also making money from ads on his lecture videos.
Legal? Probably yes, depending on university policy.
Ethical? Universities and politics nowadays encourage academics to be business-savvy and profit-oriented (I guess, as long as the uni gets a cut). So, while the described constellation indeed carries a "smell", it may be perfectly in line what society wants academics to do.
I am a college professor, and I have literally thousands of lecture/example videos on YouTube. Since I did not want any advertisement to interfere with my videos, I have not monetized my channel; however, YouTube started to add ads to some of my videos anyway. To be honest, it made me think about monetizing my channel if they put ads on my videos anyway. I have not looked into the policies the college I teach at has in place, though.
If the institution provides the server and other necessary resources to host videos, then the students are essentially paying for this through their tuition and shouldn't have to put up with ads.
However, if the professor provides the recording equipment etc., and has to arrange for the video hosting, etc. without this being provided by the institution, then I don't see anything wrong with this. It might even be that the video hosting site imposes the ads and the professor doesn't profit from them at all.
To start with a useful analogy, it has long been the case in academia that professors can assign their own (commercially published) textbook for a course, and even make this the "mandatory" text for the course. This is considered legitimate even if the professor has the power to provide a free version of the text instead, but chooses not to do so. Universities have generally taken the view that it is reasonable for students to incur basic costs (above the course fees) to obtain materials for the course, and that it is okay for the professor to make commercial profit from the material in this case. Whilst there is arguably a conflict of interest in this situation, universities have not generally been concerned about this, and they tend to see it as a necessary consequence of having experts who write textbooks teach their courses. The only time this might raise an issue is if the university draws the conclusion that the assigned commercial material is unreasonable in the circumstances (e.g., tangential to the course objectives).
Similarly, while there is arguably a conflict of interest in the case you describe, is is extremely unlikely that a university would have any problem with their professors using online teaching videos for their courses, even if they have a commercial stake in those videos, or gain advertising revenue from those videos. On most platforms the advertising is built into the platform, but even if it is something enabled by the professor, that is still roughly analogous to gaining profit from selling a commercial textbook. As with the case of an assigned textbook, the university is unlikely to have a problem with this unless they draw the conclusion that the material is unreasonable in the circumstances (e.g., tangential to the course objectives).
I am not aware of anything that would make this practice illegal --- it has been common practice in universities for generations. One could make ethical objections to the practice (on he basis of a conflict of interest), but solving this might impose bureaucratic rules that are more destructive to the quality of courses as the present practice.