First, do not assume bad faith and go talk to your advisor. There is an issue which you feel threatens the successful continuation of your PhD, the natural person to talk to about it is your advisor. In this case, it so happens that the problem involves him too, which means you have to be diplomatic about it, but he's still the right person to hash it out with. Behave professionally, do not accuse him of anything, just state the problem factually (“I need more involvement from you in order to successfully overcome this problem”), and see what he and you can come up with.
Not assuming bad faith at first is good advice in most professional situations. In this particular case, the elements you mention are:
- lack of time, which could be completely explained by other factors such as the advisor being swamped (don't get me wrong, it still needs to be fixed somehow, but it doesn't necessarily mean he's being an ass)
- rumor (“complaining about me to other faculty”), which might be just that
Now, if after trying in good faith to solve the issue with him (give it a few tries), the situation doesn't improve and it is hurting you and you think he is of bad faith: I listed several possible recourses in this related answer (and another write-up here). But don't jump the gun, because a lot of the options on this list will mean burning a bridge.