I am attending a university with a mediocre mathematics department. The courses at my school cover fewer topics and are less challenging than the same courses at top universities.
Read as many mathematics books, lecture-notes, and published articles as possible. Not just "textbooks". Not just "undergraduate textbooks" as these contain very little substantive mathematics.
Even if your faculty people are not stellar mathematicians, they surely know some advanced mathematics and can give you guidance about what to read. Uninformed self-study can be a huge time-waster so you will want some direction to be productive and your mathematics faculty will be the best & most available people to provide you with productive guidance.
Definitely talk to faculty to get guidance! This can be done officially through a "reading course" or "directed study." It can also be done informally through frequent check-ins with interesting questions. Either way this will enable those faculty members to write excellent letters of recommendation about your interest and motivation based on your capacity to appreciate the things you will have read and your perceptive questions.
It's not so much whether you (as a novice) can handle random questions, which novices rarely can, but whether your sensibilities are heading in the right direction, which would be manifest in your reaction to substantial mathematics.
As a student who just graduated from a school whose math department is outside the top 50 and going to graduate school in applied math, I offer the following advice:
Definitely get to know your professors well so they can write recommendation letters for you. One way to do that, as paul garrett mentions, is to take reading courses with professors.
It is important which classes you take: For pure math want to have at least taken classes in algebra, real/complex analysis, and topology. Note that if you plan to go to graduate school in applied math, you would want to take classes in programming, real/complex analysis, numerical linear algebra, ODEs/PDEs, and numerical analysis. Also ask your professors for advice on which classes you should take. The classes I have listed are just the bare minimum. By junior/senior year, you should be taking graduate level math courses if you plan to apply to the very best graduate schools.
Research: Although I cannot speak for pure mathematics, if you are planning to go to graduate school in applied math, try to do research with a professor at your school or a summer REU.
GRE: Most Ph.D programs in pure/applied math will require that you take the Math Subject GRE (which is not the same as the general math GRE). Although recommendations will have the most weight on your application, the subject test should not be taken lightly. In order to prepare for it, re-do problems from your calculus and linear algebra textbooks. For the other 25% of the topics on the subject GRE, look for problems in textbooks. Also try the practice tests online.
Something I really wish I had done as an undergraduate would have been to study material ahead. For example, if you were going to take real analysis in the fall semester, I would find a real analysis book in the summer and try to work my way through it by doing as many problems as I can. Doing this will make classes easier and you learn the material much better.
And if you didn't know already, https://math.stackexchange.com/ is an amazing resource for students learning math.
Good Luck!
How can I ensure that I am competitive when applying to mathematics graduate school?
Be a student member of a professional mathematics organization, such as the Mathematical Association of America, the American Mathematical Society, or the Society for Industrial and Applied Mathematics (even if you are not based in the USA). This will help you be up to date with recent developments.
Try submitting some answers to problems posed in publications like The College Mathematics Journal and Crux Mathematicorum. Publication of your name in the solutions section might help you in your application to graduate school.
If possible, attend at least one conference in mathematics. This will let you interact with students and researchers in other schools.
Have you heard about Yitang Zhang's story? If not, I would suggest you to Google for it.
Had you asked about other disciplines, my answer would be different. However, your major is math, I think top university or not does not matter much.
All you have to do is to study hard. That's it. We are Internet age now. You have all kinds of resources available online. Our sister sites, Mathematics Stack Exchange and MathOverflow, can be very helpful to you. I myself am a retiree. I learn quite a lot from them.
Bottom line, study. Good luck!
One thing that makes the top schools "top" is that they only admit the students who would get a great education anywhere, even if they had to go outside channels to find additional resources -- get involved in research, take the most serious classes you can, do additional personal projects that stretch your skills, take advantage of cheap student memberships in professional organizations, take advantage of the library including any journal subscriptions they're getting, read some of the past Masters' theses if your school keeps 'em available and well indexed...
As Tom Lehrer misquoted, "Life is like a sewer. What you get out of it depends on what you put into it."
