I recommend that you break your problem into two parts. First, how do you obtain a new PhD advisor? And second, should you complain formally, or informally, about the behavior of your current advisor? If possible, completely decouple these problems. Solve the first one first. And then decide how you would like to address the second.
Follow the rules of your university to obtain a new PhD advisor. Even if those rules require you to explain your actions, they cannot force you to fully disclose all your reasons. I would not disclose any reasons that your current advisor might perceive as personal attacks during this process. You will feel better about your situation at your university after you have a new advisor. And you can take your time to address the second problem.
The second problem is much trickier. I recommend that you start by documenting your experience of his bad behavior in considerable detail. Include all facts that support your negative experience, especially hard evidence like emails, conversations you had with third parties, etc. Write and think like a lawyer. In your mind, evaluate the strength of your documentation from the perspective of an impartial Dean at your school.
I encourage you to NOT think of complaining as an opportunity for you to vent about how badly you've been treated. Rather, think about whether you can help the university.
There are probably multiple ways you could complain about his unethical behavior. You could talk to him privately. I would only do this if you think he would be receptive to your feedback, which would be unusual. Other ways you could complain likely include a subset of these: file a formal complaint with your institution's ombudsperson; talk informally with your department's chair; file a complaint through the graduate school, either a PhD student affairs department or a Dean. Your institution may have other mechanisms. I'll use the term 'go public' to include any of these that are not limited to the advisor himself.
Note that you can talk to university administrators in the abstract to evaluate your options. That is, you can meet with them and say you've observed certain disturbing behavior in a faculty person, and ask how they recommend that you proceed, without disclosing the identity of the faculty person.
With regard about whether to proceed, weigh the plusses and minuses. It's likely that your advisor has behaved similarly in multiple situations. The situation might be analogous to recent public disclosures of inappropriate sexual behavior in entertainment and other industries.
The positive outcome that you might achieve is to protect other students from similar behavior in the future. You might also help validate negative experiences of your advisor's other advisees. The negative outcome is that you will be seen as an outlier, and the school's power structure will coalesce around your advisor.
Before you complain publicly, whether formally, informally or both, I recommend that you assess these tradeoffs by talking privately and face to face (or by phone) with people who are or have been closely associated with your advisor. Are they familiar with similar previous behavior? Did the student complain? What happened? Are there people of integrity and power who would support you? Are there other students ready to go public concurrently with you or who you think might quickly follow you?
Morally, I encourage you to go public. But practically, I would encourage you to go public only if you think the benefits outweigh the risks. It might be a difficult choice.
Finally, remember that you're under no time pressure to go public.