I think most of the good answers explain what it is to have passion, and how you "already know" if you do have one. I'll give you a bad answer, in the sense that does not directly answer the question.
I think most people have passions. But, they are called different names, depending on what they are, and why we have them. If you have a passion for math, or for electronics, than your passion can be harnessed by the society and people will respect you for that. If you're passionate about alcohol, not so much, but even then, you could become a wine taster if you go the right way about exploiting your "strength".
It is true, your passions might keep up all night, but it's so common that passions get hijacked by the very people paid to nurture them. For example, I love to create and invent stuff, I love to solve problems that seem unapproachable, I love to calculate, and I enjoy even writing the stuff down to explain to posterity what the hell I was doing. Yet, the people I work for, need me only to write long and tedious technical reports no one ever reads, about mundane research that everyone does, they need me to organize meetings and participate in committees, things that tire and disgust me, to the point I'm questioning my passion and career choices (as if I ever had those). Yet, during my vacations, I end up working on my creative stuff that might never get published, because that's the only thing I ever cared about.
Sometimes it's hard to know if you have a passion, or not. I personally don't believe people can ever be passionate about math, physics, history, robotics or whatever. Those fields are artificial, and I doubt someone was ever born with a passion for those. But, as you get exposed to these fields during your education, or self-education, you might find in the structure of one of those fields something appealing to you, some place where you get the freedom to unleash your true self. That is where your passion lie. But, it matters a lot in what way were you exposed to that field. If you are exposed to a field the way a student is, through tests and long, boring, tedious, soul crushing lectures, or the way a researcher is (or should be), through real life problems that are worth tackling.
I honestly can't say I had a passion for my field. I think I was good at it as a student because I was an overachiever, and I was interested in overachieving. When I actually started research in graduate school, it was a shock. I had to discover stuff. I had to do, not to "know" things. My stellar course work, my straight A's, meant nothing compared to the difficulty of doing actual science. It was hard and seemed not worth it at all. I almost gave up. About three years in, things seemed to connect. I started to understand what I was doing, my first paper got published, and it felt like I was good at something once more.
Two postdocs later, I realized I was doing the same type of work over and over, and didn't feel I was going anywhere, so I started thinking about quitting again. So I quit. Then I came back on a permanent position, in an awful place, true, but with a much clear idea of who I was and what I wanted to do with the rest of my life.
In a nutshell, I realized I would only care about something I created. No matter how many small problems invented by other people I solved, I always came back to my own. This is why I hated my postdoc jobs. I was doing other people's research, while my own ideas were put off indefinitely. In the name of a career and status among my peers, I was living a miserable life.
I'm not saying your passion should be research, banking, music, or your children. All I'm saying, it has to be something that truly represents you. And you will know when you will prefer to do the thing you are passionate about, in spite of the availability of cheap distractions like TV or movies, in spite of what your friends and family tell you, and despite losing your social status, or having a smaller salary.
In my opinion, the main problem with finding a passion is that the passion is hidden within yourself. Most people search for it outside. They go to art classes, they volunteer, they study in universities, go to seminars and get internships. If they are lucky, their true calling will surface during any of those activities. In most cases it never does, or goes unnoticed in the daily humdrum.
Sometimes, you can get "inspired" and you think you have found your passion because someone else who is passionate has told you about their own passion. I think it is the most common way people become passionate about things. I also think that even if you don't find your own passion that way, the "inspiration" would move you closer to it. Just listen to inspirational talks given by people who truly achieved things in their lives (I recommend the one by Arnold Schwarzenegger on youtube, because it's so hilarious). Or read the writings of Nicola Tesla. To a normal guy, they sound like madmen. In a way they are, because passion is madness. The more passionate you are, the bigger sacrifices you will be willing to make.