If the journal fails to respond to your requests over several months, this suggests that something or somebody is not working properly – even in very-long-review fields like mathematics. It may be that the editor is missing in action or that some communication got lost or the entire journal is malfunctioning, but what exactly is going on is hard to tell for any outsider.
From your account of the situation I can only see two reasons why withdrawing may not be in your favor:
The next-best choice of journal is significantly worse (in terms of level or suitability).
The paper is already under review but the status doesn’t reflect it. In this case you actually lose progress by withdrawing.
You have to decide for yourself how likely it is that one of these applies, but I consider them rather unlikely. Should you decide not to withdraw immediately, I would at least send another mail to the journal setting a clear short deadline (a week or so) for them to show at least some sign of life.