I have not even completed MS so I can't predict the detailed experience of a PhD student but before changing your supervisor, please give it a second thought.
I had to choose someone from the start of my PhD and the time is extremely limited.
You admit that you have limited time span for PhD completion and most it has already finished. Selecting some other supervisor, creating a mutual understanding and good working relationship, and completing your work atleast at the same pace at which you are working will most probably consume the time left and you may not be able to complete your thesis in the given duration.
Now lets assume, the time limit has exceeded, you'll have to face financial pressures as well in addition to the actual workload. You may get financial assistance (if you are lucky enough) but what if you don't get it? You'll have to work for managing your finances. Obviously, you'll need some time for it, which means your concentration will be divided between your job for managing expenses and your research work. Feeling bored, tired and depressed is very common among graduate students so even if you think you are really happy and contended (which you don't seem to be), with the passage of time you'll get more and more frustrated with the time your PhD is taking to be completed. So in short things will most probably get worse. (At this point i can be called a pessimist :) but still all this is possible to happen)
He is a very nice person, who is very active with public engagement, very successful at grant applications and all that.
So you are actually satisfied with the non-technical/social side of your supervisor and the major (if not only) reason for dissatisfaction is his lack of technical expertise. Lets assume, you manage to somehow change your supervisor (even without facing any political issue). What if the new supervisor is technically very sound but is non-technically too rude, proud and arrogant; not very social, takes too much time to respond to your queries, is not very good at public engagements, not really good at grant applications and so on.
You'll most probably start comparing the two supervisors in each and every single thing that happens to you. There is a big chance that you'll start missing the old one and will find some other problems in the new one. You'll regret your decision and will want to go back but most probably you won't be able to due all those financial, time, ego and politics related issues. If you continue going on in the same nostalgic mood, you'll start comparing the students/researchers working under the two supervisors and you'll probably find the better ones working with your old supervisor. In short, chances are there that you'll feel more depressed and frustrated that you are right now.
But the problem is, it turns out he does not know the subject
It may be just a wrong assumption. I totally agree with the well explained response of @Sumyrda and @Pete L. Clark
Now all evidence (corroborated by others) shows that he really is incompetent.
Don't be misguided and confused. Every one has his own opinion about the same thing and can have different experience based on his/her mindset, psychology, behavior etc. Being a student, i can easily say that there are very few students in the whole world who'll proudly say that their supervisor is actually a good one. Most students are dissatisfied by their supervisor due to one reason or the other.
At this point, you must use your own experience only to decided about your supervisor. Don't try to judge your supervisor in terms of the experience of "OTHERS". He/she may not be considered a good supervisor by everyone else in the world but still there is a chance that he/she prove to be the best supervisor for you or vice versa.
I may be wrong but i think that when you started working with this supervisor you were happy. You started expecting a lot from your supervisor and he/she was unable to meet your expectations. In the meanwhile, you tried to convince yourself that everything is fine. If in case any thing is wrong, its with you. Then at a later stage, you started discussing it with your fellows. They shared their experiences with you and you started making opinions based on their stories. Please don't be judgmental about your supervisor. He/she may actually not be that bad.
Even if he/she is, no one in this world is perfect. Every one has some strengths as well as weaknesses and we need to accept this fact. Your new supervisor will not be perfect as well. Strengths and weaknesses may vary but they'll be there.
He somehow manages to pull it off, getting on other people's papers and so on.
It seems that he tries his best to satisfy (but may be unfortunately he is not successful in his efforts). What if he or your supposedly new supervisor is least bothered and does not even try? You'll have to live and work even in that situation. At the end of the day, its your work and your job. No one else can live your life. "Accept responsibility for your life. Know that it is you who will get you where you want to go, no one else"- Les Brown
You are going to be a PhD soon and you should not expect spoon feeding from your supervisor like a FYP student. Its your field of specialization not you supervisor's so you should know more about it than him (and you'll definitely know more). Getting on other's papers is fine because no one is expert of everything.
Additionally, it seems very likely, that no one would support a rebellion (i.e. switching a supervisor), because of politics.
This part is actually serious (although most fellows here will probably won't take it seriously and ideally it shouldn't be). But lets assume you are right that your supervisor and department suffer from organizational politics etc. Changing your supervisor would really harm you. If you manage to change your supervisor (taking care of time limitations, financial, administrative and departmental issues) but your supervisor(both new and old one) are in the same department or even in same university. In such a case, you can't surely assume that you'll never see him again. If you'll have to interact with him, what level of interaction would it be? Based on this interaction, you can decide the consequences.
Last but not the least, have you actually considered alternate PhD supervisors available for switching? Does their research interest match exactly with what your work requires? What kind of guarantee can you provide yourself that your new supervisor would be technically more strong and overall better than the current one. Even if you have enough time, finance and every other relevant thing, you can't spend the whole life experiencing supervisors for PhD. Currently you are experiencing Mr. A, left him, went to Mr. B, worked with him and realized he is no better than Mr. A or is even worst than A, then what would you do? Coming back to A or finding some Mr. C would definitely be not possible