I think it's quite misleading to hold up examples of very talented mathematicians or mathematicians from the distant past. The answer depends heavily on the field you want to switch into, hold closely related it is to the field you wrote your dissertation in, and how talented and hard-working you are.
In the most classical, long-established subfields of mathematics, there is a large amount of background one needs to learn to be able to do significant, original research. To take the worst example I know, a graduate student who wants to work in the general area of the Langlands program needs about 18 months to 2 years of dedicated study after their second year graduate courses to get to the point where they can tackle some problem of interest to the research community, and this is with an advisor to guide their study and steer them away from pitfalls that would result from having an incomplete knowledge of the field. (In particular, this means the graduate student likely still has some blind spots in their knowledge that would greatly slow down their research if they didn't have an advisor.)
A graduate student doesn't only have the advantage of an advisor; he or she also has a good deal more time. Most postdocs and almost all professors have more teaching responsibilities than graduate students, and professors also have service responsibilities which increase as one gets older. Furthermore, one also has to do enough research to write somewhere between one and two reasonably significant papers a year (depending on subfield) in order to be competitive for jobs that allow time for research and eventually to earn tenure in such a job. If one wants to switch into a new field, then one presumably has to do this research in their old field while learning the new field.
If someone in representation theory or algebraic geometry or other parts of number theory wants to switch to working on Langlands, then he or she needs to learn the material in one second year graduate course (because they already studied the other two or three that a complete beginner needs) and another 18 months of specialized study. It's true that some patterns of thought will be familiar, even if the specific ideas are different, so one is going to learn somewhat faster when one learns their second field, but moving into a new field still requires at least a year of dedicated study unless someone is an unusually fast learner or extraordinarily hard working. Most people don't have the time and energy to fit an extra year of work above their other duties within any reasonable timeframe. Fifty years ago, one could have given up a couple years of paper-writing to accomplish the switch, but someone trying that today would never get another job that allows significant time for research in today's far more competitive job market.
Most areas of research don't require as much background as Langlands, but unless one is moving into an essentially brand new field requiring minimal background, switching fields requires a substantial amount of time that one simply rarely has after obtaining a PhD.
Lack of an advisor can be an issue, but it is less likely to be one than lack of time. Many fields do have a significant amount of "folklore" that is well-known to experts but not clearly stated in print anywhere. These are usually ideas that are too advanced and specialized to appear in a graduate textbook, but at the same time too easy to be the subject of a research paper. At some point this folklore is used to establish more significant results in a paper, but since "everyone" knows it, it might not be explained very clearly or be easily found by someone who needs it for some other purpose. However, most fields have experts who are quite friendly and willing to explain the necessary folklore to new entrants to the field, and, at worst, one does things in a clumsy way in his or her first papers in a new field and has some folklore pointed out by a referee. If one appears talented and capable, then it is not so hard to get some help.
Researchers in countries not connected to the international mathematical community (such as in Africa) generally face much more significant problems with having access to experts than people trying to switch fields.
Is it possible? Yes, but it's very hard.