In general, there is not much memory for individual students who act responsibly. Turning down an offer in a reasonable time period is acting responsibly. Initially accepting an offer and not turning it down until the last minute after external and internal funding applications have been made, is less responsible.
I would let departments know as soon as you know you are no longer interested. That means if you have gotten into your first choice school with a nice/fully funded offer, you should turn everyone else down. If you have gotten into your first choice school with a crappy funding offer, you need to decide for the remaining schools what you would do if they gave you a great funding offer. Any school that cannot make an offer that you would exceptaccept, you should let know.