This isn't an answer, since I don't know the French system, but advice that might help.
My best guess is that this is uncommon, since the institution you are enrolled in has little influence over an external advisor and that person has little incentive, especially financial and time, to help you when you need it. And, an advisor from another nation is probably harder to arrange than one from another French institution.
So, make sure that you have permission to do this before you start, or even commit to a program. The head of the program is probably a good source of information and can give permission if it is possible at all.
Another option that could possibly be open is to have co-advisors, one from the "home" institution and one from away, where the home advisor is responsible for keeping all the duckies lined up and the external one for the actual advice to help you advance in field.
Starting the thesis research early is also a bit risky in case you can't come to an agreement on the topic and the plan with all of the interested parties. That doesn't mean that the effort is wasted in the long term, but without a specific agreement, you might be required to change focus, putting what you have already done on hold.
Be cautious. Talk to those with authority. Get permission. Good luck.