I will relay my own experience with a similar situation.
I was TA for a lab class one semester. In the past I was TA for the associated theory class. To make a long story short, at some point the TAs for the lab class were asked to grade exams for the theory class, which I objected to as not my responsibility. I recall it was stated that the theory class TA was overloaded and we needed to help out. (That actually was false. I did my own snooping and I'd say that the theory TA would have had an uncharacteristically easy semester even if they graded all the exams.) I said that I was going to check my with advisor before grading any exams for another class. Critically, I did not refuse to do the work and merely said that doesn't seem right to me and that I am going to look into whether that is my responsibility.
My advisor agreed that this situation was fishy, and he brought it up during a department meeting. Apparently, this sort of thing has happened before. My advisor's guess is that it involves department politics neither of us are aware of. There was a meeting of the department heads and they decided to ban TAs grading assignments for classes different than the class someone is a TA for. However, they also decided to continue the practice for the remainder of the semester. My advisor told me to grade the exams, which I did on time.
The department has a TA review system. From what I was told the review I received for that class was negative, saying that I refused to perform work I was asked to. I was almost unable to be a TA again. The next time I needed to TA, my advisor and I had to talk to the professor who was handling TA assignments and tell them that the review I received was inaccurate as I did in fact do everything that was asked of me. I voluntarily did not TA again after that semester even though my funding ran out. I didn't want to deal with being a TA in my department again, mostly because I believe the time required to do a quality job exceeds the time I'm paid for.
I don't know if the false review was retaliatory or not. I think it's possible that the professor writing the review interpreted my reluctance to grade the exams for another class as refusal, and was not made aware that I did in fact grade the exams (the latter was communicated through the professor of the other class who did not review me). I agree with cag51 that it's unlikely a TA will be fired mid-semester. I do think the incident unfairly harmed my reputation in the department, at least among the few who saw the false review.
Some other recommendations:
Keep detailed notes on every meeting and action you have related to the class. I did not do this, but it would have been quite useful at times to say (for example) "On 2021-09-20 I was asked by Prof. Smith to grade exams for the theory class. I stated that seems outside the scope of my job responsibilities and that I will check with my advisor before grading those exams. On 2021-09-21 I spoke with my advisor. On 2021-10-15 I met with Prof. Johnson and the other lab TAs to receive exams to grade. On 2021-10-18 I returned the graded exams to Prof. Johnson before the 2021-10-21 deadline." Etc.
Make your concerns known in writing so that there is a paper trail, and if something needs to be communicated (for example, in my case, that I did in fact grade the exams), don't assume that it will be communicated or communicated accurately. Be explicit to avoid misunderstandings.