One mistake that you seem to be making, perhaps in your other questions here also, is an assumption that there is a single linear scale, or that all such linear scales are aligned. That isn't the case. The space is multidimensional. It isn't even completely well defined except that there are a finite number of universities at any given moment.
Basically, though, a university gains (or "tends to" gain) prestige when it does a good job on many of the varied parts of its defined mission, which is usually more than just undergraduate education. This good job needs to be recognized and usually recognized over a long period of time.
A place like Harvard is nearly four hundred years old and has been recognized in both teaching and research. It has many "Schools" with different missions. Law, Engineering, Math, the Humanities... It has a great library.
Almost every US state has one or more high prestige state funded universities. Some have several. They do a good job throughout their mission over long periods and it gets generally recognized.
A few other high prestige places, like Olin are much younger (50 years) with a limited mission, but they do a good job and it eventually gets recognized. It is the recognition of a job done well that brings the prestige, not the fact that it is hard or expensive.
With the prestige comes the ability to be selective in both students and faculty, making entry harder. If only good students and good faculty are present, then it is clearer that the place does a good job and prestige increases. The match between students and universities isn't random. Scandals, on the other hand can result in a setback.
But, for the students accepted at a "hard" prestigious university, the difficulty is about the same as at a more modest place with, perhaps, less accomplished students. So, you are unlikely to see any discrepancy in graduation rates. What you will see, perhaps, is a discrepancy of the ratio of graduates going to grad school or not, or the ratio of students going into academia vs industry.
The process of prestige building can be slow. My undergraduate institution has always done a good job but its prestige has increased greatly over the more than 50 years since I attended. It has been consistently good at its mission, which doesn't include doctoral study. Some places do a good job but its recognition is only local, so prestige may be less than otherwise. Lots of moving parts.
Building prestige (good students, faculty, research, ...) usually takes a lot of money, so the ability to attract it is a factor. Get a lot of NSF grants and prestige may go up.
Some very small colleges with a limited mission make the grade (Wellesley) because they send graduates to good grad schools and have a highly qualified faculty.
Having widely recognized faculty with a wide public presence is a big plus (Cornell, Carl Sagan).
Lots of dimensions.
