When I was on the graduate admissions committee, for each application that I personally read, I would read all three reference letters. In general I would do this about as quickly as my reading speed allowed. If there was something especially notable in a letter, I would either mark it down or orally call my colleagues' attention to it. In a substantial minority of cases I would look up the writer on the internet and get a sense of their academic profile.
I estimate that the average amount of time I spent on the "first run" through a single application is about 6-10 minutes. For the majority of the applications, a first run is sufficient. But then when we get down to ranking the candidates in any group that could potentially get admitted, we go back and reread the applications as seriously as is required to make the rankings. This could conceivably involve comparing one letter to another. Relevant passages may get read out loud which takes...as long as it takes.
A well written graduate letter is not a subtle or difficult piece of writing: it makes a clear recommendation and lays out the evidence in a way which is apparent on a rather quick reading. As a rule of thumb, if I felt the need to spend much more than ten minutes on any single letter, I would consider contacting the writer directly and asking for more information and/or a clarification.