You just described the difference between short and long term memory. Seeing something, say in a lecture, will only get your short term memory involved. To have something retained for the long term you need to actually actively learn it. Listening, reading once, seeing a video are all passive.
Yes, take notes. There is evidence that taking notes on paper is better than typing the notes as you listen to a lecture, say, because it engages the brain in a different way. Good typists, for example, can copy text without really reading or understanding it. It is just a physical, rather than mental, act.
There is brain science that shows that learning requires physically changing the stricture of the brain - reconnecting neurons. See The Art of Changing the Brain by James E Zull, for example.
But taking notes is only the first part. Practice and repetition are needed for real retention. What can you do with the information you just heard in that lecture? For undergraduates we give them exercises to reinforce the learning and make the students active in retaining it. That happens less in doctoral studies. But among other things, especially in maths you can think about and write down how this connects to other things you know and have already retained. Make the connections explicit in your thinking rather than just implicit.
When I was in grad school (maths also) I made extensive notes in lecture on paper. I still have them 50 years later and have consulted some of them recently. I do wish I'd used better paper, actually. If I were to do it again, I'd use index cards with only a single idea on each card. Better yet, take notes on good paper and just after the lecture, extract the key ideas (three most important ideas) on individual index cards. Everything here is done by hand. I did some of this actually as an undergrad (not enough) and still have a box of index card notes.
You may think that using computer based notes is superior. That is questionable. You may think it is better for retention, but it isn't for retention in your mind. Even retaining things in a computer over 50 years is problematic, since media and operating systems change as do some formats. If you back up regularly you are safer but you may also need to translate the old formats to new as many software vendors have little incentive to support really old formats. Paper lasts as long as you can keep it dry and away from fire.
