The answer depends entirely on your advisor and program/university's policies (especially the 'new' one of each).
I know of several people who changed PhD advisors for a variety of reasons: loss of advisor's funding, advisor moving to a different university, unexpected inability of an advisor to continue advising (illness, death, etc), personal conflict.
In these cases, when students had made substantial process, their prior work was included in their thesis when they moved to a new advisor within the same program. Some of these advisors worked closely with the previous advisor, or were part of the student's thesis committee.
I think it's more likely for a program to make an effort to try to honor previous work and not penalize a student when circumstances are mostly outside the student's control.
I don't know of any examples where someone transferred from one university to another and used prior work towards a thesis, but transfers between universities are rare for PhDs in general.