I was wondering if anyone can help me decide if I should ask my supervisor to be present at my Viva.
I would suggest that you should only bring your supervisor if you have a specific, positive reason to do so. If you are in two minds about it, it sounds like the answer is "no". But as your supervisor says, it is entirely your decision.
Although it is permitted to bring your supervisor into the viva as an observer, most PhD candidates do not do this. The main exception is if it would be reassuring for you to have them there. There might also be an argument for having them as a kind of "quality control", but really this is the job of the internal examiner, so unless for some reason you don't trust your internal (perhaps if they were very inexperienced), that's not necessary.
For most people, having your supervisor there would be fairly harmless, but provide no real benefit. It could be a little distracting having them in the room (given they are not allowed to say anything!) and might just mean that you start second-guessing yourself, looking to your supervisor for reassurance instead of being self-reliant. Given that it's uncommon, it's possible that it could make you look less confident in the eyes of your examiners.