I was interviewed for an academic job where I pretty much answered all the questions and the interviewer seemed positive. He even shared his contacts for further questions and said there would be more rounds of interview if everything goes well. However, at the end he said to me "All the best for your future endeavours". I am still trying to figure out what this means? Any comments in general or if someone faced a similar situation, would be nice.
"All the best" would apply regardless of whether you got the job or not. Remember that hiring isn't necessarily done just by the interviewer, but there is an additional committee with more people behind the scenes that helps decide whether you get the job or not. It's just common courtesy at the end of an interview to say things like "thank you so much for taking the time", "great getting to speak with you" (for both the interviewer and interviewee).
If you were able to answer the questions and strike a good tone with the interviewer, there isn't much more you could have done in my mind. Remember that hiring, in any case, tends to be random and sometimes involves luck. You could do everything right before, during, and after the interview and still not get the job just because they felt someone was a better fit. Don't take it personally, and just move on; who knows, maybe you may run into them in the future.
I would interpret it only as "we haven't yet made a decision" or "it isn't in my power to make a decision" but that, indeed, the interviewer wishes you well however it turns out.
Don't interpret it as a negative. I think the response would be rather different if you weren't in the running, at least. These are normally department level decisions, with a committee making recommendations to the chair or dean.
I have conducted lots of interviews in my time. I was usually the sole decider, and my mind was usually made up by the end of the interview (barring blockers in any still-to-be-done background checks, of course). My policy was to be transparent about my inclination to hire or not to the candidate (it is possible that an interviewer may get more respect if the interviewer by being austere and standoffish - showing that you are keen can be interpreted as a sign of weakness - but whatever, I always liked to be straight with people). So coming from me the parting remark would have been a definite goodbye forever.
What others have said about it being a routine parting wish or even a non-committal close to an otherwise promising dialogue is possible.
But whenever someone wished me well in the future at the end of an interview, it never happened that I was offered the job. It was often accompanied by a warm smile (always the only smile during the whole process) and hearty handshake too which always aroused my suspicion that the interview had become a tedious pro forma exercise after initial impressions were taken or bad ones confirmed - and the interviewer was relieved to see the back of me.
In your case of it being an academic job, your interviewer was most likely a senior academic rather than from HR -- the latter being notoriously non-committal with everything they say at interviews ! - so I'd keep going on the job-hunt if I were you: you're unlikely to be more than a fallback for this one.
Sounds to me like that interviewer is not likely the person who ultimately makes the hiring decision, but rather only has input into it. They might have thought you did well and were implying that they hope you're seen favorably by the hiring authorities above them.
If they were implying that you didn't get the job, that would be a bit like turning down a date by telling the person that they have a great personality. A bit too soon, harsh, and obvious in intent.
This is essentially a fixed phrase for rejection in a hiring process, so much so that some employers likely avoid it even when rejecting an applicant to sound more friendly and personal.
An interviewer who is sensitive to topics like this would avoid this phrase in all circumstances other then to unofficially inform of rejection.
That being said, some interviewers may not realise or even purposefully ignore this for whatever reason, such as an attempt to sound neutral in their own way.
Overall, I don't think it is a definite rejection, but I would be suspicious that this may be the case.
As an aside, if there are multiple factors causing you to feel that the hiring manager may not be high in emotional intelligence, it is up to you whether to consider it a red flag.