University-wide reputations are somewhat of a self-amplifying phenomenon:
University-wide reputation is an established thing. This is opposed to the reputation of a department or faculty.
Students and academics want to be at a well-reputed university. Thus, well-reputed universities can be more selective and get better students and staff.
Better students and academics at reputed universities raise the level of teaching and research through existing qualities, higher teaching levels, cross-fertilisation, researchers having access to better students, etc.
Reputed universities actually are better as they produce better graduates and research, attract more funding, etc.
University-wide reputation becomes a reasonable criterion and gets further established. Go to Step 1.
However, without such a process, university-wide reputations hardly makes any sense:
Most interactions in universities happen within departments or at least within faculties, and thus there is no reason to assume the qualities of different faculties correlate with each other.
For example, the idea that a university with an excellent science faculty also must have a good law faculty is absurd (except for the above mechanism).
In some countries, such a process has happened; in Germany, it mostly didn’t.
Germans rarely think about the reputation of a university, but rather about the reputation of a department, and there are indeed strong variabilities between those.
This is at least partially due to historical factors, but there also some systemic factors:
In the German education system, students specialise on a field upon entering university. This reduces the interactions between departments as compared to other systems.
German universities are primarily funded by taxes not by fees and donations. People do not make donations to their alma mater; they pay taxes. Reputed universities cannot raise higher fees. There is no rich-get-richer amplification furthering the above process.
The German culture is rather egalitarian and particularly holds to the ideal of providing free and equal education to everybody (or in case of universities, everybody with certain prerequisites). The concept of an elite school or university is not generally well regarded. (Mind that whether those egalitarian ideals are actually achieved is another question.)
A considerable amount of research in Germany happens at dedicated research institutes (mostly Max Planck, Helmholtz, Leibniz, and Fraunhofer Institutes) that are usually only loosely affiliated with universities. Whatever reputation these institutes acquire does not fully rub off on universities (in public perception as well as methodic rankings).
Note how the German universities that have a somewhat generally good reputation tend to be located in beautiful and expensive cities and thus are more attractive to students with rich parents (which despite all egalitarian tendencies have better prerequisites). Finally, mind that there is a (disputed) initiative that may initialise the above process, but even it does, I would expect it to take decades to show effect.