I doubt that the reputation of the author has a large effect in general, especially at the review stage. It might if the paper suggests an especially important result. In that case, a quick read and the likelihood that it is correct will motivate the editor and reviewers to get it finished quickly. But the sheer volume of submissions suggests that the effect is small, and rare.
More likely, delays are natural. If the topic is obscure it may be hard to find appropriate reviewers. If the result is a small contribution, there will be little incentive to advance it over others. Sometimes the paper requires additional checking, if it isn't especially self-contained or if the techniques used are non-standard.
Moreover, the editor doesn't have a lot of control over the work habits or schedules of the reviewers as they are usually unpaid volunteers. Academic schedules get in the way of doing things quickly as does the too common absent mindedness of some professors. Sometimes the reviews come back to the editor in a way that it is hard to make a decision and the paper is sent out to others so that the editor gets a clearer picture. Sometimes reviewer conflicts need to be addressed internally.
Once accepted, the delays are usually just caused by scheduling. For print journals, especially, where overall page limits put in a constraint, as well as the fact that some editors may want to bring similarly themed papers into a single volume. At this point, of course, it may be desirable to think about the reputation of the authors and so an unknown author with a minor result might get delayed a bit.