Having been on the graduate committee of the departments I am/was in for the past six years, and having looked at around 500 applications over that time, I've got something to say about this:
The time it took you to graduate doesn't matter. 2 years is fast, and if you've got good grades, then you're clearly quite good and that speaks to your case.
If you're only doing the bare minimum of classes, then that raises questions about your preparedness. That's no different than the many applications we see from small liberal arts colleges whose math departments are so small that they can't run anything other than the standard set of courses. The question then always is how such candidates will do in advanced classes, given that they don't have as much background in the different branches of mathematics as applicants from larger schools. There is also the question of the level of courses you've taken: Many applicants take graduate classes in their last year as undergraduates, and if they do well there, that is definitely a plus on their applications.
You can always apply. If you get rejected, that's ok, and it won't be a stain on your file if you come back better prepared next year. But you'll have to have an answer to the question of what happens if you get accepted into a program of lesser quality: Are you going to take it, or hold out for something better next year. Declining an offer definitely will look bad if you try again next year, but of course there is also no guarantee that you'll actually get accepted at a better program next time around. So there is a risk involved in all of this.
At the end of the day, your best approach is probably to consider what happens if you graduate now, get accepted into a good program, and then find that you're not adequately prepared. Grad school is fast-paced, and you're thrown into the same pot as all of the other students, many of whom will likely have taken more (and more difficult) classes than you had -- in many cases, first year graduate program classes. You'll struggle -- I'm pretty confident saying that because essentially every graduate student struggles, and the question is only to the degree with which they do.
You can get ahead this curve a bit by staying for another year at your current institution, taking advanced courses, maybe a graduate course or two, and doing well. Your file is going to look better for it, and you'll have better chances of getting into the good graduate programs. On the other hand, your file is not going to look noticeably worse just because it took you three instead of two years to graduate -- your competition will have taken four years. The only real downside I can think to staying for a third year is the money you will have to pay for tuition and cost of living.