First, you should consider: what is your motivation for making this switch, and why do you think you will be successful? Is there some compelling need to make this change (such as limited job prospects in field X, or a better lifestyle in field Y)? Do you have the key aptitudes at a sufficient level to be successful in field Y? Have these aptitudes been objectively measured? And finally: are you already familiar enough with the state of field Y to know that it is a better fit? Note that by being familiar with field Y, we mean having some concrete experience such as a research project or a class, not just anecdotes or guesswork. Remember, you are comparing a field you know very well to a field you know less well, so it may be that the grass just seems greener on the other side.
Our best advice is to talk to experts in your field, especially those involved in hiring for field Y. Things vary widely by field, but even more so by candidate: switching from (say) theoretical physics to pure math might make sense for one theoretical physicist but not for another.
We will also emphasize from the beginning that drastic switches between fields are to be avoided when possible. Beginning students sometimes have complicated plans to start in field X, then switch to altogether different field Y and eventually end up in field Z. Such plans are rarely realistic. Switching fields certainly may be doable, but it is rarely an optimal path. In particular, choosing field Y simply because it is “hot” at the moment is rarely a good choice. Being hot implies that there will be a lot of competition and from people likely better prepared than yourself at the moment.
We will break the remainder of our answer into three cases.
X and Y very closely related
It is probably not necessary to formally “switch fields”. Rather, you can simply continue your current research in X while also adding collaborators in Y. As lighthouse keeper wrote: “Have your fingers in several pies. Build up some new expertise while working on some low-hanging fruits from your existing background that will lead to immediate results.” Indeed, as Dave Clarke wrote: “Some shifting of topic is not only permitted, it is required.” Switching to a new topic will let you expand your network, gain new skills, and think about new problems; this is a natural career progression.
The hardest part may be finding your first position in field Y, as Buffy tells us: If you have already finished your PhD in X and are searching for a post-doc or professorship in field Y, it may be difficult to compete with candidates who are already experts in Y. If you are not willing to take a position in X and slowly transition toward Y, you will likely need the support of a professor in Y. Once you have publications and contacts in field Y, no one will care that your degree was actually in X.
X and Y less closely related
This is more difficult. As Wrzlprmft tells us, you need to identify a reason for the PI to hire you rather than someone who already is familiar with Y. There are basically two options: either your background in X will give you a unique perspective on Y, or there are not enough “homegrown” candidates in Y and so the PIs will need to be less selective. The latter also indicates that you are moving into an expanding field, which is likely to offer good job prospects later in your career.
Note that it is generally not necessary or advisable to do a second PhD. Remember, a PhD does not teach you everything there is to know about X; rather, it teaches you how to do research in X. Your understanding of the fundamentals, competence in relevant research methods, and ability to rapidly get smart on new topics should enable you to switch from X to Y without needing further faculty supervision. And there is generally no need for a formal degree in Y, as quarague tells us: it is sufficient to state that you have a PhD in X and a strong background in Y.
In some cultures/fields, it is common to take a short master’s degree, summer school, or “boot camp” in field Y to come up to speed and make some connections. The UK, for example, offers conversion courses, as Sammy Gerbil tells us.
There are also some instances in which universities will try to “redeploy” staff such as post-docs at the end of their contract, and this can often involve field switches. For example, Ian Turton describes having run a computational geography group in the UK that would “snap up chemists, mathematicians, and anyone else who could think and write papers. We have loads of people to teach them geography when they arrived.” However, this is far from a global norm; in other cultures, post-docs are expected to make their own arrangements for what to do when their contract expires.
X and Y completely disjoint
Again, you should consider whether there is any overlap between these subjects. Even apparently distinct subjects like law and religion have considerable overlap. This will allow you to continue your career in X while also becoming active in Y. Interdisciplinary work like this can be very interesting.
But in some cases, this is not possible (or is not aligned with your interests): there is not much overlap between particle physics and 17th-century Dutch paintings, for example. In this case, you should really think about what switching fields means to you. Do you really intend to apply for professional research positions in field Y? If not, there is likely no need to formally switch fields; you can simply pursue your interest in Y during your spare time.
But if you really do intend to apply for research positions in field Y, you probably will have to start over, starting with a second PhD. As Scott Seidman tells us, this will raise some concerns, for example about your decision making or the possibility of perpetual-student syndrome. So, you should be prepared to address these concerns. All students should think carefully about their long-term career plans (particularly given how few PhD graduates will succeed in finding a tenured position), but this is doubly true for students beginning a second PhD.
Please feel free to edit this answer to improve it. Please add content to this answer rather than posting additional answers.
