Maybe a dumb question, but for example: say I'm doing a PhD in Theoretical Physics (I'm not) but I need to have taken certain undergrad math courses to understand the material, which I never took in undergrad. Do grad students ever end up in undergrad classes for this reason, or do they just learn the material themselves out of books?
As others say, this depends entirely on specific institutional culture and policies. But from my experience (studying/working at various universities in North America and Western Europe):
You can almost certainly audit the class — i.e. go to the lectures, get access to online study materials, etc.
You can probably, but not certainly, take the class — i.e. submit homeworks, take the exam, and get assessment, feedback, and a grade. At some universities, especially in the US, this may incur tuition fees, or count towards an allowance of how many courses you can take. All programmes I’ve been involved in allow this, as far as I know, but I wouldn’t be shocked by an institution that didn’t.
You may or may not be able to use this course as credit towards your PhD requirements. Most PhD programmes require you to take a certain number of courses, but have certain limitations (in both level and topic) on which courses count for this. Some programmes I’ve known have allowed higher-level undergrad courses to count; others haven’t.
There are some institutions where auditing is more expensive than enrolling in a course for credit; often this comes from losing the tax money from the government agency. There are some institutions where there is a pass-fail or pass-no record option; in some cases, a grade of D will either count as a fail, or be reported as a D. In some cases, the professor turns in a letter grade and the registrar changes it to pass, fail, or whatever.
At other schools, there is a Math 505 or some analogous number which is only open to non-math graduate students and covers what they missed in their undergraduate program.
And taking a course for as long as you can and then withdrawing? It may not affect your GPA, but Financial Aid may view that the same as failing the course.
In order not to influence any committees that may decide to pass you or offer you a fellowship, I might ask the main graduate school office of the university(ies) you are interested in, rather than the department, for their answer.