This may vary a lot depending on the field, university, and advisor you end up with, but I'll answer according to what I've seen (in Applied Math, in the U.S.).
No, this is not at all true. Even once you get accepted to a PhD program, you usually have qualifying/preliminary exams you need to pass within your first few years (with a limited number of attempts). If you don't pass these, you are gone from the program (though you are often allowed to leave with a Master's; I would say that is often a consolation prize).
Once you pass the qualifying exams, you have to find an advisor willing to work with you, a research topic, and pass some sort of comprehensive exam that is related to your topic. Some advisors are picky about who they work with, so it is possible to not find an advisor. It is also possible to fail the comprehensive exam (you usually do not get a second attempt).
Once you have completed those hurdles, you have to actually write a thesis, and have your advisor basically approve it as being ready to defend. And then you have to defend it. I have seen people who, despite years of work, never get their thesis to a point where their advisor approves them to defend. I think there is typically some sort of time limit on your thesis from when you start the program, but this is according to university regulations. I have also seen unsuccessful thesis defenses; these are rare (since usually you just will not be allowed to defend if the thesis is bad), but if it happens you are gone without a PhD.
I suppose that being persistent could include applying to a different PhD program after being kicked out of a first (or a second). Maybe if it is a low enough ranked program they will give out a PhD to anyone who sticks it out, but I don't really know about this.