Disclaimer: When I first read the unedited version of the question, I assumed that, since OP recently finished his/her PhD, they more or less figured out how to write papers during their studies, and that the question was more about an authorship conflict.
Should I submit without him?
You seem to present two options: wait for your advisor to review, possibly modify, and approve your manuscript, or submit the paper as the sole-author. It is not entirely clear that you should be rushing things on your manuscript at this point, whether or not you truly believe that your advisor deserves co-authorship, what the advisor co-authorship norms are where you are at, etc. Anyway, moving on ...
Unfortunately, I cannot say which course of action is the best one for your particular situation. As with anything, it helps to understand the consequences of any potential actions you take.
If you submit your paper without resolving this conflict with your advisor, then that could irreparably damage your professional relationship with him. This, in turn, could lead to no support from him for your future career endeavors, such as in the form of letters of recommendation, networking and collaboration opportunities, etc.
If you are OK with the above outcomes, then, if you do decide to submit, you may face some additional challenges. As highlighted in an answer by user Peter Jansson to this question, if your advisor decides to contact the journal, then this could result in a long, drawn-out dispute. You may emerge from the dispute with a paper, with either you as the sole author, or with you and your former advisor as co-authors. You may also emerge from the dispute without a paper published in that particular journal at all.
From your post, I get the sense that you are worried that your advisor is dragging the review process out needlessly. However, if you are willing to entertain the idea that your advisor deserves to be a co-author, my "advice" would be to attempt to rein in your advisor's manuscript review time, by sending them frequent reminders, that you are going to submit by such-and-such date, etc. And, who knows, you might end up with a stronger paper as a result of his feedback.