This hasn't directly happened to me, but it's come close. Basically I meet a professor for other reasons (I attended a talk by them, I requested a recommendation from them, during a social visit to their department, etc), and we talk about their work. They are clearly very passionate about their subject, and they eventually offer further collaboration with e.g. "We have this open position and are looking to hire. Would you like to apply?"
Problem is, I don't find their work very interesting. What is a good way to say I don't intend to collaborate further because I don't find the topic interesting? What if we are already collaborating, but I intend to end the collaboration because I lost interest in the topic? Does it matter if the topics (i.e. the one I work on / would like to work on vs. the one the professor works on) are different niches in the same subject, or if they are completely disjoint? For example, if I'm a number theorist, do I respond differently to a professor in algebra or to a professor in medicine?