I have found that for some reason people really dislike it when other people do things for reasons that aren't "noble".
I posted a question on here a while ago asking about doing a post-doc, and I said that the main reason I would want to do one is because it seems like it would be a fun way to spend my time for a few years. I got a lot of negative replies along the lines of "If that's the reason you want to do a post-doc, you clearly aren't the type of person who should be doing one" or "I hope no one wastes research dollars on you".
Similarly, anyone who creates a start-up with the goal of "getting rich" is harassed. If they had pretended their real goal is to solve some great problem in the world in order to make it a better place, they would have gotten a lot of positive feedback instead.
In the programming community, a lot of people say "you're not a real programmer" if you don't have side projects that you work on in your free time. How can you possibly be passionate about your work if you don't do it in your free time as well?
In my opinion, all of the above is a lot of baloney. Who cares what your motivation is. We're all going to be dead in the next hundred years, and the only thing that matters now is the tangible results you can provide to your employing agency. If I can produce high quality research publications in a post-doc that I'm doing for "all the wrong reasons", who cares? Life isn't fair; there are people who slave away and have little to show for their work, while others hit upon a lucky discovery with little effort and are set for life. If you would rather spend your free time drinking beer and socializing with friends instead of reading articles, but you're still able to make good progress on your PhD, who cares?
I've found that most people who brag about working hard and being at the office all the time are normally wasting their time in other ways anyhow. Sure, they may be "working" 10 hours a day, but they're not actually accomplishing all that much.
So I wouldn't let other students' opinions on whether you're "passionate enough" have any bearing on what you do with your life.
EDIT: Having said that, I will point out that probabilistically, those who are more curious and interested in their work will tend to do better than those who aren't. If you're forcing yourself to work 60 hours a week, you will probably not do as well as someone who has to stop themselves from working that many hours because they're having so much fun. For instance, I spend my research time performing molecular dynamics simulations, writing programs to analyze the data, and writing papers on the results. But then I spend my free time learning about quantum mechanics. Maybe what I do in my free time could somehow help my research; then again, maybe it won't. But I don't get burnt out, because I'm splitting my time up into divisions of what I can handle without anxiety.