As an assistant professor, you would have four constituencies: undergrads, graduate students, your department's tenured and "sure thing" pre-tenured faculty, and the school's administration.
Undergrads want to know what's going to be on the test. If your age isn't going to be on the test, they won't notice it. To them, a professor is a professor is an old person. Just be prepared and predictable and they'll be happy. Being 40 might command more respect than being 30, and should mean fewer crushes to fend off. I consider it a plus for this group.
Grad students want you not to be insane. "Dear Lord, please allow that my advisor be not changeable like the wind, be not stubborn like the glaciers, and be not crazy, and be not on drugs. Also I would be grateful if you could manage that she not work me like an indentured servant, please, and thank you." They'll judge you by your recent publications, your connections and intelligence.
Some or all of the faculty will be writing reviews of you and your research program in three or four years. For that reason, you should talk to them about theirs. For real. They can judge your acuity better on home turf than when adrift in a sea of unfamiliar words and ideas, and they'll be flattered. If you came in as a 23-year-old wiz-kid, there might be some resentment or envy. If you are pleasant and dedicated to your career, irrelevant factors like your age will not undermine your prospects with your new colleagues.
Presidents and Deans want to hear only good things about you. The Dean will be told of any difficulties you experience, and could possibly alert the President, so don't have any. They might not even know your age, and I can't think of a reason they'd care about it. Help the department move up the prestige ladder and they wouldn't mind if you were 60.
Age could be an issue when you apply for jobs. They'll assume you were 18 when you started undergrad, regardless of when you graduated. If you graduated late, you could put the year you graduated instead of the years of attendance. Your age might be off-putting for the hiring committee before they meet you. If they get over it and invite you to interview with them, you'll have your CV, references, and many chances to make a good impression. You haven't been playing mahjong for the last few years. As long as you "read" like you're of the tribe, your age will not be a negative in academia.