I was in a very similar situation. My advice is to take it on the chin and finish. In a couple semesters this will be behind you. You will never have to speak to this guy again.
Understand this. Your: from the perspective of your department, your professor is an asset to the department. You are a liability. There is no 'slap on the wrist' for him. There is a 'he cant work well with others' FOR YOU. Now, of course your department might be filled with bigger character integrities than mine (iI certainly hope this is true), but the black and white is that you are an invited guest and the department is one meeting away from conspiring to get you out the door with not even a 'good luck' to show for it. He is valuable. You are not.
Everyone wants a phdPhD until its time to phdPhD. Take a big whiff. This is what it takes. God bless you, sir. You are not alone.
Edit.: Here is what you should do. Approach a third party within your discipline. Do not mention you're having problems with your advisor. This third party should be a relatively established respected memebermember of the department. Say something along the lines of "please, "Please join our regular group meeting as iI am approaching graduation and iI would greatly appreciate extra eyes and feedback on my work". A group dynamic should help reduced the toxicity. It is imperative they join as an extra observer. Do not mention any personal problems. Do not take no for an answer. Do not attend one on one meetings with your supervisor.