Your answer will depend partly on what you want to do in academia. If you want to teach, but don't really want to emphasize research much, you might do fine to graduate now. However, if your goal is to become a professor at a research intensive school, then you really should go to the strongest grad school you can get into. (Based on your description, I strongly suspect that if you bust your butt for another year or two, in particular working to earn one or two strong letters of rec, you could get into a better grad school than you can currently.)
Yes, I know there are considerations about who you will work with, perhaps geography, potential two-body problems, etc. So, why's it so important to go to the best school you can? Again and again I see that in academia (as everywhere) networking is crucial. Generalizing and stereotyping a bit: the best schools have the best researchers, who know the other best researchers, who have the biggest grants, which fund the nicest postdocs, etc. If you want to thrive as a researcher, you will do well to get into that network. (To a large extent, it's a rich get richer system.) As an undergrad or early grad student, one way you can get into that network is to work with a professor who is a central part of it, and is willing to weave you in. And your chances of working with said professor typically go up with the reputation of the school.
Now a personal digression. Through high school and undergrad I was in a hurry to get to the next level as soon as possible. I skipped 7th grade, finished undergrad in 3 years, and started grad school at the age of 20. I even turned down a year abroad in the Budapest Semester in Math, because I was worried I'd miss out if I waited too long to get to grad school. The sad truth is that I wasn't ready. Maybe you would be; I've never met you, so I can't say.
Eventually (after 8 years), I finished a PhD, and am fairly happy with where my career is headed. However, I don't regret that time in grad school at all. I learned a lot of useful stuff. In fact, I think it's because I took my time in grad school that my career has gone as well as it has. One interesting feature of academia is that you're typically judged by your productivity relative to the time since you earned your PhD (rather than your age). As a result, I encourage you to take your time and learn as much as you can. You'll never again have as much free time as you do now.