I have seen plenty of thesis defenses that use blackboards (the boards in our department are black- rather than white-, but I don't see that it makes a difference) and also plenty that have used slides. You should ask around -- or remember from previous thesis defenses you have attended; these are almost always open to all interested parties -- to see whether there is any local preference between slides and writing on the board. Ask your advisor too.
I suspect that you will find that it is up to you. In that case: do what makes you feel comfortable.
Is it a bad idea to use whiteboard and markers instead of slides in my Ph.D. oral defense?
No, if you know it will make you more comfortable and more able to present well, it's a good idea.
If I use whiteboard, what if the committee have questions about the contents that I have already erased in my presentation?
I don't see any fundamental difference between a thesis defense and any other math presentation. In general people can ask questions at any time; if the speaker indicates that she would rather address them later, then they get addressed later, in which case it's up to both the speaker and the questioner to remember the question (e.g. someone could write it down). A question which is specifically pertinent to something complicated you wrote on the board is probably better addressed then rather than later. If you get a later question about something that's not on the board anymore, your options are to answer the question verbally, to rewrite something on the board, or to pass out some other written materials which address it.
I will mention in passing that mathematical culture seems to prefer "chalk talks" more than general academic culture at this point. I have absolutely never used slides / powerpoint / whatever in any class I've taught. I do it only at large conferences where I can't be confident in advance that the board will be available and easily visible to all and/or that I've been given such a short amount of time that I feel that I have to whip through slides at top speed in order to "cover" what I want to. Writing mathematics on the board takes time. But so does understanding mathematics, and it feels a bit dishonest to need to present material faster than you can write it on the board. Moreover, if you're writing then you can adapt your presentation to the audience on the fly (which for a thesis can be important, so that e.g. having a thesis defense in a room with nothing to write on at all would seem weird to me).
Further Thoughts: Many thesis defenses have two very distinct parts, and the first part really is a lot like a conference talk in that you have a set amount of material to "cover" and the general expectation that you will get through that "coverage" with little distraction. So doing slides for that part of the defense makes perfectly good sense to me: a thesis defense is not a lecture, and the goal of that part of the defense is not to impart the maximal possible understanding to everyone in the room. You want to get through that first part with little fuss, and you don't want it to drag out. So for instance if you are the type of person that prepares much more to write on the board than turns out to be realistic (as I often am), then preparing slides could be a good strategy to stay on track. Still, if you want to write on the board during this part of the defense: go for it.