[Background: I'm a full professor in a top-5 American CS department. My answer is skewed toward standard practice in theoretical computer science, but I regularly give the same advice to all incoming PhD students in my department.]
As others have said, practices vary in different departments, and in different sub-fields within each department, but as a general rule: It's your PhD. Of course you can switch advisors!
Professor A voted for your admission, but he did not admit you. The department admitted you. Prof A's vote does not oblige you to work with him at all. It's natural for Prof A to anticipate that you will work with him, but he has no right to demand that you work with him. (These are all shadings of the word "expect"; I'm not sure which meaning you intended.) If you prefer to work with Prof B, you do not need to justify your preference to Prof A. (You should still communicate your preference to Prof A, of course, especially if he asks you about working with him.)
More generally, you should never feel constrained to only work with one professor, just because that professor happens to be your advisor or happens to be paying you. Nothing prevents you from working with both Prof A and Prof B, provided you are meeting the obligations attached to your funding. (Exclusivity is not a reasonable obligation for a research assistantship. Progress, yes. Good-faith effort, definitely. Results, sure. Exclusivity, no.)
In my department, faculty in my research area strongly encourage entering PhD students to work with multiple faculty — in the first year, to help everyone make more informed advising decisions, and in later years, so that students develop a more diverse research portfolio, and to provide a backup in case research with one faculty member doesn't pan out. Yes, even the students I am supporting from my own grants.
All that said, it's best to approach the situation under the assumption of good faith. Assume that Prof A and the other faculty in your new department have your best interests at heart, and approach them to discuss options under that assumption. Be clear with Prof A that you are not sure you want to work with him and why, and ask for advice on how best to meet your professional goals, including approaching other potential advisors. If Prof A objects, or tries to guilt-trip you into working with him, or tries to convince you not to even talk with other faculty, then congratulations, you've identified someone who does not have your best interests at heart. Best to learn that early, before you waste your time working with them.