I agree with other answers that, in general, it is defensible either to hold class on such a day, or not hold it. However, your question is different because you're asking it now, less than a week before the day in question. I'd like to make the point that, if you are making the decision at this late date, I think you are honor bound to do something to compensate the students who would have attended because they have already planned their shcedule in order to do so.
Many students arrange rides, trains, plane flights, etc., to go home for Thanksgiving, and if they know they have class on that day and thought it was important, they may already have configured their schedules to attend. As someone who has not forgotten what it is like to be a student, I can say that it really sucks to be in that situation and then have the professor cavalierly cancel class, leaving you (the student) in the position of having postponed your trip home for no reason.
Realistically, the only way you can do this is to hold class and make it worth attending. You could do this by making it "fun" (although I think that has to mean more than just "a cool topic" -- at least bring muffins or something to reward the diehards) or, preferably, by making it genuinely useful. Depending on what the class is about, you could spend some extra time on a difficult topic, perhaps go through some example problems (if it's that kind of class), so that those who attend will get extra practice that will actually help them in the class. If your syllabus always clearly showed that class was scheduled, and you haven't given any hint of it not being, it could also be defensible to hold some sort of trivial "pop quiz" that would give a few extra points to those who attend. (You can find other questions on this site with opinions on the ethics of this, but if you have reserved a portion of the class grade for attendance or participation, this is the time to use it to give people a bonus for showing up.)
In short, in general it is defensible to cancel class on a day when few people are expected to show up, but if you do that you have to telegraph your intentions early on. I don't think it's acceptable to cancel class for such a reason on less than a week's notice, when students may have already arranged their schedules based on their belief that class will be held. To do so is unfair to students who took you at your (and your syllabus's) word, and penalizes exactly the stalwart and upstanding students who made their plans in order to be able to go to class as scheduled, while rewarding those who had already made the decision to play hookey.
I do think, though, that you could possibly announce what you are doing on that day. In other words, you could say, "Oops, I forgot about Thanksgiving. We'll still be holding class, and it will be [whatever -- review session, quiz for participation points, etc.]." This will make it clear to the students who are coming that they are going to get something out of it, and also make it clear to the students who aren't coming that they are going to miss something that will actually be relevant to the class and aren't getting off with nothing. In a way, this can be a good litmus test for whether what you're doing on that day is legit --- if students who already planned to skip have a decent chance of thinking "Oops, that might have been helpful, oh well", then the class is meaningful enough to compensate the students who do attend.