In evaluating tenure track candidates in mathematics, everyone is paying very close attention to the year of the PhD. This is because you are hiring candidates not primarily based on their current position but on their perceived trajectory: you want to hire the gal or guy who will be a research star five or ten years down the line. (At Harvard, Princeton and a few other places, they want to hire the person who's a research superduperstar right now. But they would only hire someone young if they truly thought that their present accomplishments were tops on a worldwide scale. Anyway, this is quite an exceptional case...)
Beyond that, it's all arguable, and the arguments have a strange way of favoring the candidate in one's field, who one knows / wants to work with anyway....In all seriousness, it's really not clear. When I evaluate tenure-track candidates I look at their productivity both in an absolute sense -- i.e., total output -- and in a relative sense -- i.e., rate of production. The person who is two years out and has eight papers of course looks better to me than the person who is four years out and has eight papers. They also look better to me than the person who is four years out and has ten papers (assuming that the papers and the fields are comparable, which they may not be). But the person who is two years out, has three papers, two of which are amazing and joint with their PhD advisor and the other one is really strong too but seems roughly similar? Ideally you'd like to see a few more data points, but if you really think you could grab them right now....At a certain point it becomes openly game-theoretic.
One thing I'll say is that a greater percentage of top candidates are slightly older than the "standard efficient route" would predict. A lot of people get their PhD in math in their late 20's, and if you're especially sharp, went to the right places, and didn't mess around you could certainly get your PhD by 25 or so. You might think that the belles of the ball are the candidates who got their PhDs at 25. And some of them are, but for each of those there are a couple more who are really strong but somehow managed to get their PhD past the age of 30, either by transferring from a foreign country, doing military service, or simply spending six-seven years in grad school (and writing a very strong thesis).
Moreover, there are a very small number of postdoc (often called "X Assistant Professor") positions at very prestigious places that last for longer than three years. If you can spend more than three years in one top postdoc, then you have received the advantage of an essentially ideal environment for maximizing your research productivity, and thus it should not be too surprising that a lot of these people go on to very good jobs. But if instead of spending five years in one postdoc you spend five years in three postdocs, that's much less auspicious: though there are exceptions, most people can only spin the wheel so many times before they cash out with whatever they can get (and we hirers know that, having in many cases been there).