Computer Science PhD admit here.
Is it crazy of me to turn down a top school (MIT/Stanford) for a lower-ranked school (think Cornell/UW/Columbia/Michigan) if I think there is better advisor fit?
Obviously all of these are fantastic schools and I am very lucky to be choosing from them. However, I felt like I got along really well with my would-be advisor at the lower-ranked school than I did at MIT/Stanford. People have told me that it's crazy of me to turn down MIT/Stanford since they are, along with Berkeley/CMU, on a different "level" than the other schools.
(My field is somewhat narrow so there is only 1~2 faculty at each of the schools doing research in the area. So regardless of which school I go to, if the advisor doesn't work out, I would be in a dire position.)
EDIT: Both advisors are well known in the field (and both have produced outstanding graduates), but the MIT/Stanford professor is good deal more senior. Also, talking with their current students, the MIT/Stanford professor is known to be pushy and have a "strong" personality (which may work for some people, but not sure whether it will work for me, as I've never worked with such people before).