This probably varies by field. I went into an Engr/CS department. I will say that grad school apps are a bit of a crap-shoot at top schools. There are certainly "talent picks" (e.g., people so amazing many schools would grab them without knowing where they'd fit), but in my experience top programs often have labs that have particular needs and "draft" PhD candidates who might fill them. If you expect to be funded with a stipend through grad school (and in my opinion, in STEM, you should be), somebody in the faculty needs to step up and allocate their limited grant funding on a candidate within the first year.
Since your tuition is going to be paid by one of the profs on the faculty, there's obviously going to be some level of preference toward students whose interests and skills match their needs. Hence, part of your likelihood of being accepted (and most of your probability of being funded) depends on who will have slots for PhD students and how those slots match up with your interests/skills. Or, in other words, you need to convince somebody that you are worth at least $70k over 4-6 years (in reality, more than that, as professors invest their time in training you to be useful).
In that light, you're really not applying for a "top grad school." You are actually applying to one or two labs in a grad school... with no idea if they can even take you on board. In my opinion, your statement in your application should directly reflect a lab (or two) that you would want to do work with. The whole remainder of your materials help support the claim that you would be an asset to those labs. This also means that you generally need to apply to quite a few grad schools if you want to get into a top one. Unless you know a program closely already, there is no way to know who has funding coming in, who is on sabbatical, or a hundred other things that could sink an otherwise solid application.
Obviously, good grades, good GRE scores in relevant areas, and prior research experience are all your first foot in the door. But none of them are enough, on their own. You need to write well (e.g., have a good statement), have great independent work ethic (e.g., leading teams, not just working on them), and your letters of reference need to be excellent. You'll also need three letters of reference at almost any top school. I don't think I applied to a single place that took less than that (though that was some years ago, so it could even be up to 4 in some programs!). If you can't find three profs who will speak highly of you, you're in trouble. If having a personal reference vouch for you even crosses your mind, don't even bother applying to a top program.
Finally, if you're applying to MIT or a few other places, you had better keep track of all the texts that you have ever used. They ask you to list them. Which is a huge pain to try to rustle up 3 years later, for the record.