It is possible to do a PhD part time in France. Or rather, it is possible to have a full time job and do a PhD on the side. You will not get a part time doctoral contract. This is common in social sciences or the humanities. In math, this is also common when school teachers want to get a PhD for various reasons, e.g. teach in a university or in a higher preparatory school (or just for the heck of it).
One of the main downside is, of course, that your PhD is going to take much longer than the three years standard. Six years would be a more reasonable target; PhDs can take a decade to complete in humanities.
Let me mention that self-funding is not an option. If you do not have a doctoral contract (aka PhD-related funding), doctoral schools will ask for proof of income.
But you should reread your potential advisor's email. From what you have told us, it does not appear to say that he is willing to be your advisor if you do a PhD part time. This is a completely different promise than saying he is willing to be your advisor if you find funding. You should clarify this with him but I am unsure that you will like the answer.
The best person to ask about funding opportunities is your potential advisor. Without knowing details (what university, what field) it is impossible to answer beyond vague generalities. This is too dependent on local factors such as how many stipends do the ministry or local governments give out, companies that may be willing to cofund PhDs, local fellowships, and so on. You can ask your potential advisor again, but if the first reply was already that they do not have funding, then I would wager that you are out of luck.
