This is an unfortunately common situation that a lot of highly skilled professionals find themselves in, not just academics. I know from personal experience that it's not easy.
Still, I would say that you either take the job or you don't. Continuing to interview with other places after making this commitment is unfair to the institution who's willing to go through the trouble of hiring you. It's also unfair to yourself and your partner, because it extends the state of uncertainty, making it hard to make plans and move forward.
It's a risk, there's no doubt about that. You can easily find horror stories of people whose visas were denied for weird or grossly unfair reasons. But you also have to ask yourself whether looking up such stories is helpful for your mental health or for making a decision about the position. If possible, ask the university for their success rate with visas. Ask them if they have a back-up plan if your visa gets denied. And keep in mind that they are taking a risk too, and by making you an offer, they've deemed it worthwhile.
If you decide to accept the job, you should definitely inform the other institutions that you're off the market. However, you don't have to cancel the interviews outright. You could ask them if they're willing to chat regardless, just to build the connection. They can say no, but some might be intrigued and want to talk anyway. (This actually happened for me, and it turned out to be a nice chat.)