For very standard, orthodox material based on a textbook, it is reasonable to not take notes and do as much engaged, active thinking-or-questioning during a lecture. One should be alert to insights (hopefully!) from the person in the front of the room, and from others, so a bit of note-writing about the peripheral things is to be expected.
For advanced courses, and for seminars based only loosely on publicly-available material, or actually intended to be _explications_of_ otherwise-opaque material, the task is to both take as many notes as possible (even if/when printed material is provided), and think as much as remaining resources allow, because without notes the words spoken and written will mostly vanish. Here I overlook the possibility that one's memory is so excellent and so practiced that one truly can perfectly remember things one does not understand. The latter possibility is very important to cultivate, but this question wouldn't have arisen at all if that were already in reach.
And, yes, in advanced courses and seminars, although I've gotten over the surprise, I am baffled at the claim that people can't usefully take notes. The usual claim is that by not taking notes they think about the material in real time. This would be great if it were usefully true, but I find that my students do not have total recall... so that mostly they have neither notes nor recollection.
Perhaps the main practical trick to learn is to be able to write, very fast, without looking at the paper, and be able to "copy" the visual layout of the blackboard (whiteboard, whatever) without necessarily stopping listening to the audio. Yes, this does require a lot of effort, but, hopefully, it gains something.