Short version: I started a PhD with an advisor who I am no longer interested in working with. Can I still leave this lab and find another lab in the same field (how)?
Long version: I am an American student at an American university. My PhD advisor, who I met during my masters, told me he had two projects and I could be on whichever one I chose. Project A was fully funded and in an area that I was very interested. Project B was not fully funded, would require me to TA, and the research area was not that interesting to me. I specifically told him I would come to his lab if I could work on project A. He agreed and so I worked on this project for over a year. The project is massive and involves lots of other labs from the same field all working together collaboratively. There has not been a time when my advisor expressed disapproval of my work. Recently, my advisor tells me I am going to be working on project B now which can only fund me part time. He also hired a new PhD and promised him he could be funded by project A like he did for me. I asked him why this sudden change. He simply says not to "worry about it" and I will be fine working on project B. So I guess I got baited? Also, since I have joined the lab, my advisor's demeanor has changed from friendly to somewhat aggressive with some of the other students, however he has not targeted me yet. Still, I don't want to keep pressuring him for an explanation because I am afraid that I may get on his bad side.
I would very much like to leave this lab. I don't have any interest in project B and I can't help but feel that my advisor is a bit of a shyster. I spoke to some other faculty about it at the university. They offered to write me a LOR in my applications to other labs. This is because it is highly unlikely my advisor will respond well to me announcing that I am leaving based on how he has responded to other "bad news" in the past. I wouldn't expect him to write me a LOR.
I was also told that there is no way that a new lab in the same field won't know about my history with my advisor, and they likely won't accept me because of it. "A graduate student is not worth crossing another professor". Unfortunately, it's true. When I apply to other labs in the same field they will certainly see my past work with my advisor. He is well known by this community and they will certainly reach out to him.
My question is this: Is there any way I can continue in the field of research which I have already spend thousands of hours studying? Or do I have to completely start fresh just to get away?
I have seen some threads opened up about changing labs (I am sure I could probably find a new lab in a completely different area of research). However, None of these threads seem to have addressed the issue of staying the same field. (Except on Quora which says it cannot be done)