(I speak from limited experience, and none on the "reading applications and deciding" side. Technically my undergraduate degree was in a program somewhere between math and CS, but I'd taken a fairly math-oriented slant on it.)
Having a masters degree in a related field certainly doesn't make you ineligible to pursue a PhD in math. The only disadvantage I can see for taking time to do a masters degree in a slightly different area, as opposed to, say, working in industry, is that you may need to work harder to explain why you're changing areas.
(Also you'll have some record of doing research, and you might be judged in part on the quality of that research where an undergrad might be given the benefit of the doubt.)
If you want to continue working in a closely related area that's a bit more in the mathematical direction, it shouldn't be very hard to explain why you've decided to continue that work in mathematics rather than CS. It's a bigger issue if you want to do something very different in math---the worry would be that whatever caused you to lose interest in computational algebraic geometry will happen again. That's not insurmountable, but it will be more of a hurdle to convince the faculty you're really interested enough in the new area to stick out a PhD.