"I'm currently studying undergraduate physics at Cambridge UK (and a UK citizen). However, I've found the lack of rigor in the math we use extremely frustrating. I've dabbled in math texts (Spivak's Calculus and Axler's Linear Algebra Done Right) and absolutely loved proof based math, though I appreciate this is at a first-year level."
I agree with this answer: You will have more opportunities for exposure to what you call "proof based math" in the more senior years of your undergraduate studies, and if that's not enough then in Part III you can select courses from anywhere among pure mathematics, applied mathematics, statistics and theoretical physics (the latter including highly proof-based courses like quantum information, which is taught more as a math course than a physics course). Plenty of people do a PhD in pure math or applied math after an undergraduate or masters degree in physics. Also, Cambridge students have unusually long break periods between terms, during which you can study more pure mathematics on your own as you have already been doing, while still remaining in the natural sciences program which makes you much more likely to secure a job (whether in academia or elsewhere) after you get your degree.
"Does anyone have advice on how to switch to mathematics after a physics degree?"
Take mathematical courses that are available to you during your undergraduate physics degree, continue reading mathematics books like the ones you mentioned, and ask people working in mathematical areas if they are working on any projects in which an undergraduate research assistant (you!) can participate. Be careful that you're really doing the right thing for you (it seems that one year ago you thought that physics was the wiser choice, and now you've changed your mind, so keep in mind that it can change again, and continue talking to as many people as you can in both fields: peers, students more senior to you, professionals, professors, advisors, etc.).
"I've already seen suggestions to pursue a Masters or Bacc in the US but the issue with that is I couldn't pay for it."
The US isn't the only place that offers Masters degrees. Your own university has Part III of the Mathematical Tripos, and plenty of universities and institutes in Europe, Canada, etc. have Masters degrees with scholarships or lower costs than universities in USA. In Canada you usually get paid to do Masters degrees, but at your stage it might be better to aim to get into a PhD program, since you can switch from a PhD program into a Masters one if you want to (or have to) exit early.
"Is "catching up" during the first couple of years of a math PhD an option? I've seen even highly competitive programmes such as Yale mention that it is possible to take remedial undergraduate courses during a PhD."
Yes. Catching up is what almost all PhD students do at the start of their PhD program, and taking undergraduate level courses as a PhD student is pretty much universally accepted.
"Alternatively, can anyone recommend a route through physics that would allow me to pick up more rigorous math? I've also looked at physics PhD programmes but these tend to require a broad base in physics that I'm slowly losing interest in."
You can take mathematical courses whenever you have the option, continue studying rigorous mathematics during the unusually long Cambridge break periods, and you can try to work as a research assistant with mathematicians (I would be surprised if your college didn't send you information about the research assistantship program that we sent towards the end of Lent term 2023 by the way, since more than 50 Oxbridge students responded).
"Edit: the UK system means unfortunately switching majors/taking extra years as an undergraduate isn't possible. The closest would be completing the "Part III" (masters) math course but this tends to restrict physics students to courses such as QFT."
No, in Part III courses are offered in:
- Algebra,
- Algebraic Geometry,
- Analysis and PDEs,
- Combinatorics,
- Differential Geometry and Topology,
- "Foundations",
- Number Theory,
- Information and Finance,
- Probability, Statistics,
- Quantum Computation and Information and Foundations,
- Particle Physics and QFT,
- Reltivity and Cosmology,
- Applied and Computational Analysis,
- Astrophysics,
- Soft Matter and Biological Physics, and
- Continuum Mechanics
"Students may select courses freely from those available, subject to the constraints of the lecture timetable, regardless of which department they registered with or which application stream they applied through." so you are not restricted to taking QFT (Quantum Field Theory) and similar courses.