In brief: find a different advisor. If this is difficult, then your department should have a “Director of Graduate Studies”, or some similar title, and they should be able to help you.
Your situation as you describe it sounds odd in several ways. The only reasons I’ve heard of for having an advisor in a different department are if you really want to work with that advisor, or at least in their field. But it sounds like that’s definitely not the case here! So it’s not clear that there’s any good reason for you to be working with this external advisor, rather than someone in your department.
You mention in comments that one potential advisor in your department has previously rejected you. If it was only one, then there should be others to ask, reasonably close to your areas of interest; hopefully, one of them would be happy to take you on.
If you find that multiple faculty members in your department are unwilling to work with you, then the situation is more difficult. There could be many different reasons — perhaps your department has too many PhD students at the moment, so faculty are over-stretched; or possibly the potential advisors feel that your work is not as promising as it should be. But in either case, the director of your department’s graduate programme should be able to help understand and resolve the situation — part of their job is ensuring that no student ends up without a suitable advisor.