In mathematics, there are certain open questions which are the subject of current research (think of the newest papers in the top journals). I have also heard that some graduate programs ask questions such as these on their qualifying exam. How common is this practice? What kinds of questions can I expect to see on a qualifying exam? I know that a qualifying exam typically tests the knowledge gained in graduate school, but I'm not sure to what extent this knowledge is tested? How "advanced" should I expect these questions to be?
I think you shouldn't believe everything you hear. Mathematicians like to tell stories about posing various famous open problems to certain brilliant young students, the idea being that these students don't have the respect and fear for these problems that an adult mathematician would and that maybe, just maybe, they will innocently work on them and even solve them. And it seems to be true that every once in a great while a student solves a serious open problem this way. The vast majority of the time they don't, of course: a problem that the experts in the field want to solve but can't is not going to be accessible to a young student, however brilliant, 99.99% of the time.
So maybe every once in a while a faculty member decides to put a "ringer" problem on an exam. What do you think is going to happen? With probability very close to one, no one will solve the problem, and the writer will have to own up to the fact that the question was not an appropriate one and make sure the students were not in any way penalized by its appearance on the exam. How many times have I seen this happen on exams I have taken, written, or otherwise encountered? Zero.
In fact qualifying exams test a standard syllabus; they certainly do not test research level mathematics. Every department does their exams at least a little differently (a very few departments, like Princeton, don't even have written qualifying exams!) so I don't think there's much point in preparing / worrying about quals until you enroll in a specific graduate program.
Conversely, when you arrive in a graduate program, you should start asking for information about the qualifying exams. And you'll get it: most programs have posted syllabi for quals. Many programs, including those I've personally been involved with, have years and years of old quals for you to page through. Nowadays a lot of these are freely available online: in fact, please see Ohio State's webpage: not only do they do they have posted all of their own qualifying exams going back at least ten years, they helpfully post links to qualifying exams at almost 20 different departments.
Thus the way to study for and pass quals is to go through old quals learning the material and techniques that come up again and again until you can do most of the problems on a given qual in a reasonable amount of time. It's not really any more mysterious or romantic than that...