There's a disconnect between the title, the actual question, and the tags you've chosen for it.
Here, I'll break it down for you.
As a male student, did I say something wrong to this female professor?
The word "wrong" is ambiguous. There's both a pragmatic meaning to wrong ("Did I do something wrong with respect to the goal of getting along with people and advancing my career?") as well as an ethical meaning ("Did I do something morally wrong?"). These aren't the same thing. For example, in the business world, being a corporate whistleblower is sometimes bad for one's career and not very conducive to getting along with people, but it's morally virtuous and good for the economy as a whole. On the other hand, telling the boss what they want to hear, instead of the truth, can sometimes to be good for your career, but it's morally dubious and it's bad for business and it's bad for the economy as a whole.
A professor and I spoke recently (I'm a grad student), and she told me
about her upcoming sabbatical, and I wished her well and was genuinely
excited for her and her travel plans. But the reaction that I got from
her was one of caution and displeasure, with no reciprocation of good
vibes.
People are strange sometimes. I wouldn't worry too much about this.
I'm a male student. Have I possibly said something condescending or disrespectful to a female professor, by showing my excitement about her travel plans?
Suppose we're talking about ethics. Then we should ask the question: is it always morally wrong to be disrespectful? Assuming the answer is "no," then even if your enthusiasm was construed as disrespectful, this still fails to establish that you're morally in the wrong. Ergo, if the other side wishes to establish your moral wrongness, they will have to make a rather precise argument. For example, it might be of the form:
In [specific set of circumstances], it is always morally wrong to perform an act that ends up being construed as disrespectful.
We were in the aforementioned set of circumstances and you performed an act that was construed as disrespectful.
Therefore, the aforementioned act you performed was morally wrong.
To my mind, this seems like a pretty tenuous line of thinking, to say the least. My point is that even if the other side did construe your actions as disrespectful, this is still a long long long long long way from establishing the moral wrongness of those actions. And, I wouldn't worry about the morality of your actions in this context. In my estimation, your actions seem perfectly ethical.
Suppose, on the other hand, we're speaking from a career advancement perspective. Well, it's definitely bad for one's career to perform actions that are construed as disrespectful by the people who are holding the reins of power. However, to some extent, you can't tell what will be construed as respectful or disrespectful. Sometimes, you will be friendly, and this will be construed as disrepectful. Sometimes you will be respectful, and this will be construed as cold and unfriendly. I think that the basic rules, for those who wish to be perceived as respectful, are:
Be friendly, but don't be whimsical, flippant, silly, absurd.
Don't exaggerate.
Don't come off as sexist, racist, or otherwise judgemental of the other person's socioeconomic background.
When people speak, listen to what they have to say.
If you disagree, let them finish speaking; don't interrupt them.
If you disagree, use phrases like "I respectfully disagree," and "I strongly object to...," which make you sound more civilized and worthy of being listened to.
Again, this has very little to do with morality; it's just a bunch of tips & tricks for appearing respectful.
It sounds like you may have violated (2) somewhat, insofar as your excitement may have seemed exaggerated or feigned. I see no reason to think (3) was violated, except perhaps in the other person's mind, and purely as a corollary of moderately violating (2). It's (2) you should be worrying about, not (3). But of course, people are strange, and even if you follow the above dot points, you will still sometimes offend people. I recommend getting used to it, since this will probably happen again, and the alternative to getting used to occasionally offending people is basically a pathetic life lived in fear, which is not something I can, in good conscience, recommend.
When I analyzed it a bit, I came to the conclusion that I would have had the same reactions if a male professor told me of his sabbatical, travel plans. So, I don't think that I've said anything sexist. But I am not 100% sure.
Once again, people are strange sometimes, and I wouldn't worry too much about this. But the idea, implicit in the question and it's tagging with "ethics" that there's this thing called, sexism whose moral wrongness is somehow pre-established and beyond question, is in my very honest opinion deeply suspect. The word "sexism" is thrown around a lot, but in practice a lot of the people who use it can't define it, and those who can define it, usually define it so broadly (e.g. "discrimination on the basis of sex") that it becomes impossible to argue in favor of the position that sexism is always morally wrong, or otherwise use it to establish the moral wrongness of any given act. All they can really do is point out that certain sexist acts are morally wrong. Holding women's mathematics to a different standard than men's mathematics, for example, is certainly is certainly morally wrong, but this can be deduced from general principles of fairness, and more to the point, giving a single example of sexist act that is morally wrong fails to establish anything like a universal moral wrongness of sexism, and therefore fails to establish any viable general principles from which to infer the wrongness of any given act. So, although I do recommend avoiding sexism if you want to advance your academic career, I respectfully object to the position, implicit in your question, that there's this unambiguous thing called sexism (which there isn't), and that all instances of it are morally wrong (which it's not clear that they are).