(I'm not familiar with what overlearning exactly is, so I go by what Wiki says it is)
Depending on the field of science there are things where overlearning (as in vocabulary) is beneficial or even needed. E.g. I'm chemist. Unfortunately, there are priciples behind chemistry that cannot be explained like you explain how to derive a formula. But knowing lots of these things allows you to extrapolate to new situations. During undergrad studies we all complained that if we had wanted this style of studies, we had chosen medicine and started learning the phone directory by hard.
It is a bit like learning a language like children do: without being told, structures will emerge.
These structures of the knowledge that form by experience are also important for topics/techniques that can be explained in a logical way. They allow you to become fast and efficient. They allow to "automate" certain tasks, and that leaves conscious capacity to think about less well-known things. In physics, it will certainly be beneficial if you have your intellectual capacity free for thinking about physics because you mastered the relevant maths to a level where you see possible ways of solving instead of digging through your brain and notes for that. Ultimately, this level of mastering subjects is also what is needed to arrive at truly new connections ("Good mathematicians see analogies. Great mathematicians see analogies between analogies." - true for physicists and chemists and whole lots of other researchers as well).
This means you need to exercise the stuff you learned. Doing textbook excercises is a boring (but possibly efficient) way of doing that at intermediate levels. But of course, you can try to do this with research projects as well. The important thing also with undergrad research is that you train your research/physics skills and not only do what you are told to do, but also take the time to understand why things are done this way. What alternatives would exist, what would be the disadvantage of that etc.
About studying ahead of schedule: in my university (Germany) it was no problem to participate e.g. in written exams ahead of the "regular" time. If you passed, noone asked you to go to the course. In some subjects, it even gave you the benefit that if you wanted to try again when your year was doing the exam, you could redo it and pick the better mark. Also, noone would have kept you from attending advanced lectures/seminars. Or if you went for subjects that were not strictly on the plan, like another language. I also heard some philosophy, took some computer science and some economics and an introduction to general and patent law - you get the idea. We did not do much research that way, but that was probably because we had research projects on schedule, so the research groups had rather more research students than they could handle (I don't count fetching stuff from the library as research job).
