From browsing through many threads on Reddit and discussing with people offsite, I find that the general impression is that most German universities are equally good; for example this one.
How did they become equally good? As far as I understand, there is a high degree of self governance for the German state governments to choose the policy in regards to their universities, so I'd imagine a higher variance in quality by state.
There a few factors here:
There are different reputation-quality-feedback loops in play. To oversimplify slightly: In the US, people believe that universities have signficant quality differences. Thus, good students and academics go to highly ranked universities. This leads to highly-ranked universities to actually be signficantly better, which feeds the belief that this is the case. In Germany, people don't perceive significant differences, so they don't care about ratings. Subsequently, good students and academics are more evenly distributed amongst universities, and quality differences are small.
German universities have limited means to attract/select the best people. Salaries are determined by the state, as are the required teaching hours. Admission requirements can't be overly strict by law either. This again leads to a more even distribution of who goes where in terms of talent.
Germany has a culture of proper job qualifications other than university degrees, and thus far fewer people go to university. It seems about 1/3 of German school leavers go to university, while it's 2/3 in the US. As such, German universities are all comparable only with the more selective US institutions as far as incoming student quality is concerned.
Asking for whether a German university is better than another makes roughly as much sense whether craftspeople in one city are better than in another: Sure, some businesses are better than others, one city will have the very best craftsperson, and maybe one of the cities has a vibrant plumbing community leading to a significantly better quality of plumbing in that city. However, at the end of the day, this will have little effect on the average quality of a services or help you select a particular service: If you need your violin repaired, it doesn’t help you that the world’s best hair stylist lives next door.
Similarly, the idea that a university is generally better than another is – prima facie – absurd: If a university has one of the world’s leading historians or is best at teaching biology to its students, this tells you very little of the quality of teaching or research at its math department. You may have a strong correlation of quality within an institute, a modest one within a department, but this becomes negligible once you go to the faculty or university level. The main thing that connects different faculties within a university are its central services (administration, cantina, etc.), but these barely affect teaching and research quality.
Thus I think the question rather is: Why do other countries have such high difference in university quality? I think the main reasons are:
Reputation feedback loop: A good reputation attracts better students and personnel, which in turn leads to better teaching and research, which leads to a better reputation. More on this in this answer of mine.
Financial feedback loop: More money allows you to hire better researchers and teachers, have a higher teacher-to-student ratio, and perform more expensive research, which in turn allows you to acquire more money from student fees, sponsors, and grants.
Historical bias: For example, in the US most (if not all) top universities are rather old or were funded with insane amounts of money (see the previous point). Young universities usually have bad chances of acquiring a reputation etc.
As German universities are state-funded, the financial loop is mostly absent. Also, this limits the reputation loop: Why should the state fund more room or personnel at one university when there is unused space and personnel elsewhere? As for the historical aspect, Germany had many universities before it became feasible for most students and personnel to select a university solely based on quality aspects as opposed to geographical ones. Those were of comparable quality for the same reason outlined in the initial example with craftspeople.
As far as I understand, there is a high degree of self governance for the German state governments to choose the policy in regards to their universities, so I'd imagine a higher variance in quality by state.
Kind of. On the one hand things such as finances are largely homogenised, so they don’t factor into this. On the other hand, for other aspects so much freedom is left to the individual faculties, and departments that the effects smear out at the university level. For example most physics departments in Germany adhere to rather similar standards of teaching, but this is completely different from how any law department operates.
I don't know a lot about the US system but I guess a lot of the top unis there are heavily funded by private citizens and companies. I guess the salaries of professors is also not uniform. I am also guessing that "good" unis get grants more easily, can afford more high-end state of the art labs and equipment that help them to produce more cutting edge research, which in turn makes them even more attractive and their "betterness" is structurally perpetuated.
In Germany, on the other hand, (public) university education is paid for by the state and the amount of money that the universities get is roughly the same for similar amounts of students. The salaries of professors are regulated by the German government as well - every professor at the same level gets more or less the same money. So there is not much reason to have a "brain drain" of the best professors to specific universities - the benefits are not as big. Of course, some unis are more attractive / have a bit more prestige (like e.g. Heidelberg or Tübingen due to being very old institutions, or unis in bigger cities like Hamburg or Berlin due to the added value of being able to live in a great city).
Also, if not many students enroll in your programs because the quality of learning is bad, you might get less funds as a university, which is an incentive to keep the quality of education as high as possible.
I think it makes sense to think of this question differently: why do US universities have such differences in reputation?
Comparing apples to apples, German universities do have big differences in reputation. The institutions in the US that primarily play the role of Universitäten and those that primarily play the role of Fachhochschulen are both called universities. Universitäten and Fachhochschulen definitely do NOT have the same reputation in Germany.
The US systems are designed with differences in reputation built in. By law, the various campuses of the University of California accept the top 12% of California high school students. The remaining California students are simply not allowed to go to one of the University of California campuses; they go to one of the California State University campuses. (Note all these campuses pretty much operate as independent universities, and, in practice, UCB and UCLA have students in the top 4% and the other 9 UC campuses have students in the 5-12% range.)
(Think about this question in the French context too - it's obvious why ENS has a much higher reputation than the ordinary French universities.)
