(I speak for a mathematics grad program and admissions...) In any case, it's not literal "ranking" that is generated by some for-profit group, but, rather, not-so-young-anymore faculty peoples' knowledge of the relative rigor and standards of a program, and the appreciation (or lack thereof) of that faculty's conception and awareness of what graduate programs demand of students. Of course, yes, some faculty have arrived at modest places by choice or by chance after times spent at "high-status/tier" places, so have an idea about the larger world, so their appraisals of the potential success of their students carries more weight. Yes, obviously, some faculty at lower-tier places have less idea of the rigors of "fancier" places, so give dubious opinions about future success of their students. It is more plausible to imagine that faculty at higher-tier places are aware of how their own graduate program works, at least, and can imagine the success-or-not of their undergrad proteges in comparable programs.
In summary, it's not really about "status/ranking" per se, but about "demands of the program", and expectations for PhD projects and coursework background.
In particular, it is not the case that while looking at grad applications, I automatically think applicants from higher-tier schools are better... What does tend to be the case is that there's less volatility in the letters-of-recommendation from such places, since the faculty have more substantial track records (regardless of whether they're big-shots), and can speak more convincingly about probable success of their students in our program.
For applicants from small colleges, etc, it is indeed a bit harder to gauge potential, because they may not yet have been challenged, by coursework or projects. (Many or most "REU"s are of interest, but don't at all reliably "stress" participants, and this is by design, presumably...) Not a moral failing, for sure, but not so useful in gauging potential for future success in a less jolly, more stressful, environment. But, duh, admissions committees realize that not all mathematically talented people happen to go to colleges/universities where the faculty are able, or inclined, or allowed (!) to stress them. And maybe the available courses are limited, for various reasons.