I am a professor in a top ten math dept. in the US. I have sat on admissions committees for many years, and talked to colleagues at other top tier institutions about admissions.
First: admissions is not done by a "bureacracy", or "officers". It is done by some subset of the math dept. faculty, who read the applications, including the letters, the GRE scores, the transcripts, and the essay, and then rank order them.
Second: To get to a top institution, high GRE scores are essential. These don't guarantee admission by any means, but if someone can't get high scores on this exam, it calls into question their understanding of all the basic undergrad math they've learnt. It's true that different schools (and different faculty members, even at a given school) place different weights on this exam, but doing well on it is something you have some control over in the admissions process (by preparing well), so it makes sense to do so.
Your GPA in your math courses is also very important. Presumably you are doing your best in your courses, and getting as high a GPA as possible. So there is no real magic to this; you just have to work hard at learning math.
Third: Assuming that your GPA and GRE scores put you in the ball-park of being a credible candidate for admission, people will read your letters carefully. So you want to get letter writers who can write about your achievements and abilities in as much detail as possible.
Fourth: No-one expects undergrads to have done real research; REU experience certainly helps, but one main reason for this is that it provides a way to meet professors who may get to know you quite well, and so can (hopefully) write a strong letter for you.
Fifth: Graduate courses certainly help, if your grade in them is good and meaningful. It is often the case that undergrads in grad courses will be given somewhat inflated grades out of sympathy on the part of the instructor. This makes sense from the point of view of not destroying someone's GPA because they took a challenging class, but an admissions committee will look for evidence that the undergrad really did master the material in the grad classes they have taken. One way to show this is to have the instructor of the graduate class write a (hopefully positive) letter.
Again, different schools have different expectations about what incoming students will know. At the absolute top places (Harvard, Princeton) essentially all the incoming students will know essentially all the material in basic graduate courses (measure theory and functional analysis, basic algebraic topology, basic commutative algebra, and so on). At other places this is not the case, but most incoming students at most top schools will be familiar with a reasonable percentage of what would be regarded as basic graduate math.
Sixth:
In terms of preparation for grad school, writing a senior thesis is great. It teaches you about a topic in much more depth than you would normally investigate it, normally leads to learning some grad level math, and also again builds a closer connection with a professor (your advisor) who can then hopefully write a good letter for you. And at a more fundamental level, since mathematicians spend most of their time writing about mathematics, this is pretty good preparation for that.
As far as applications to grad school go (rather than as preparation for succeeding in graduate school, where as I've said it's great), my sense is that the real pay-off for writing a senior thesis is the letter from the advisor.
Since this is a bit harder to quantify than entries on a transcript, often students are advised to take grad classes rather than write a senior thesis (if this option exists). I can see why this advice is given, but I do think writing a senior thesis (if you put your heart into it) is an invaluable mathematical experience.
A related option is to do independent study on a topic with a faculty member. Again, from the pragmatic point of view this doesn't do much for your transcript, but the benefits for your mathematical education are similar to those of writing a senior thesis, if you have a good advisor and take the independent study seriously.
Finally: Remember that there are lots graduate schools in mathematics. What I've written above more-or-less reflects admissions experience at top ten institutions, but obviously there are many more very good math programs out there, and they are not all as competitive as the top ten.
So overall, the most sensible thing to do is to work on learning as much math as you can as well as you can, and interacting with your professors enough that they know what you're learning and can write about it. I think the only things that are really worth thinking about in terms of "gaming" the admissions process are attending an REU or doing independent study or writing a senior thesis, something that gives you a chance to interact with a professor, and a specific topic, in more depth; and making sure you study well for the GRE before you take it.
