Overview
I am a physics PhD student, currently in my fourth year, in a US university. During the last couple of years, I have been experiencing a decrease on the respect I have for my advisor, for a series of reasons. Besides all the stuff related to his role as a teacher in our institution (mostly his lack of effort to make students learn), there are several reasons related to his role as my advisor.
The Problems
He has a very hard time to explain his ideas clearly, and even harder time showing (I mean actually writing down on a blackboard) mathematical relationships containing the information that he's very verbose about. This makes me feel that his knowledge about the details of the topics he works on is somewhat limited. He doesn't offer much structured guidance as advisor. He jumps between quite different areas in short time scales, pursuing only hot topics that (in my opinion) could maximise the number of citations received per unit time spent on preparing a paper, while he's not an expert at all in many of those topics. The depth of our papers is thus not great, although the yield is (we've published ten papers).
When I mention that I'm not happy with my understanding about some part of a paper, or about the confidence I have on the claims we make, I usually receive a dissuasive answer suggesting me to look at it in more depth after the paper is published, because it's important to send the paper as soon as possible.
When I mention that my research work seems unstructured and it feels hard to think about how to put everything together for a thesis, he seems not to see that as a valid point. Indeed, I often receive a "this goes in the thesis" when we're starting to work on a new paper.
What I consider to be the worst is that we have very different ideas of what academic ethics means. His approach to our projects is like "we need to obtain a result X for the paper to be relevant, so let's try to take only data that is favourable, let's only explore the model to the range of applications that could give favourable results, and let's not mention clearly if we obtain an unfavourable result".
Of course, I am a co-author of those papers too, so I am responsible for it as well. My own (soft and not too confident) personality doesn't help in going against his will. He's quite pushy and not often open to changing his mind. He speaks with grandiosity, but usually doesn't back up his words with technical details. I am not in the best mental condition, so many times I just give up after my opinions or suggestions are dismissed a couple of times.
Facing the second half of my fourth year, I'm trying to decide how to continue. There is a topic I am interested in, but I have lost faith in the ability of my advisor to guide me through it. I have been studying, and I already have experience with many of the concepts, and the tools (mostly simulations and statistics for analysis) that I would need to actually do some interesting work. I have mentioned three times my interest on focusing on this project as my dissertation work, instead of collecting all the previous papers and making a dissertation out of them. He seemed to understand and even respect my idea when we talked about it, but his actions haven't changed. He keeps pushing me into new projects and not caring about my desires. I find it hard to go to campus and meet him, and I try to avoid it unless I have to teach my own classes. I have very negative feelings towards him these days, and I feel very disgusted by the idea of doing my dissertation in his style. I think if I did that, I would just not have any respect for myself or my work, and would end up getting out of academia.
Questions
I would appreciate some advice on how to deal with the situation, or comments about similar situations you have gone through.
The specific questions I want to ask are:
To which extent am I supposed to submit to his will in terms of what to work on for a dissertation?
How feasible is it for me to direct myself on this specific topic I want to work on? What alternatives could I use, such as contacting other researchers who could provide some guidance?
Do I make a dissertation with my advisor without feeling good about it, or do I make it without my advisor and feeling better about it?