I was wondering, I'm interested in both some aspects of pure math and some aspects of theoretical computer science. I'm currently in a PhD program in computer science at a pretty decent institution (let's say, top 5 in most TCS people's minds), but I think my interests are more mathematical than I had anticipated. (For example, I'm realizing I'm more interested in some areas of topology and geometry and connections to computer science.) Do you think it would be worth trying to transfer into the math PhD program/do you that would be viable? I'm just about to start my second year. It would likely set me back a year at least to pass the qualifying exam in the math department, but I sort of think that it would do me good to acquire a more robust background in math. Of course, this is a very individual consideration, but I wanted to get a sense of what the general feeling on this type of question would be before chatting seriously with faculty in the two departments. (I know that usually people move from math into CS, not vice versa...)
I can't balance your options for you, of course, but I'll give you a couple of things to consider if you haven't already.
One is the time to completion of your current degree project. If it is soon, then staying might be best and then moving gradually into your math studies, either formally or informally.
But another option than the one you mention would be to ask around to see if you can't find an additional advisor in math and work with both of them at the intersection of math and CS which you seem to be interested it. This pretty much assumes that you are not yet deep in the weeds of your CS doctoral research. You might be able to avoid the math qualifiers that way if you have already qualified in CS.
I would guess that there is already some collaboration between the departments at an institution such as you describe. You might be able to explore it painlessly without needing to make an immediate decision or burn bridges.
What would be on your credential when you finish would depend on a bunch of things, of course.
The first year is sort of a calibration period, so unless the stakes are too high, some fluctuation is to be expected. I usually advise people that dropping out or switching chairs during the first year is a much better option than being stuck for 3-4 years with a topic (or advisor) you found is not for you.
PhDs are hard as nails already, so why risk burning yourself out on a topic that doesn't match your interests. Matters like the level of your institution or in which direction people usually move seem secondary in this situation.
Regardless of what advice you get here though, being you i would go and talk to your advisor about this. Simply be open. They will more likely have a specific idea how to proceed, and who knows, maybe you can work out a pretty novel direction by involving a co-advisor.