As mentioned in the comments, the answer is that there are typically no age limits. What postdoc positions occasionally limit (e.g., NSF postdocs) is the number of years since you've gotten your PhD, and there is often a preference to hiring recents PhD, which is what we mean typically mean by "young mathematicians." (We usually look at dates of degrees rather than age specifically when reviewing applications.)
Many mathematicians started off in other fields first, and there are quite a number of successful mathematicians who have gotten their PhDs in their 30s-40s. See this MO question for examples. Also, here is a related MSE question.
To be honest, I haven't seen enough examples personally from the hiring perspective to know whether there is some bias against age in hiring postdocs or not--I imagine there is at some point, e.g. at 60 or 70, people may not think it's worth training someone who may not have any good years left and such a candidate would need to have done something truly amazing to be competitive. But at 40, this is not as big of an issue as it might have been in the past--many mathematicians nowadays do great work after 40 (including people on the MO list above) and are recognized as such. This is corroborated by the fact that research universities regularly hire professors who in their 40s and 50s (though it is rare to hire someone to do research someone who is already in their late 50s or 60s). So, I wouldn't worry too much about a bias against your age.