I had the same exact problem and I then later found out I have dyslexia and dyscalculia. I am not in anyway suggesting that you have the same issue, but I was diagnosed very late in life (18 yo) and I honestly see many colleagues having the same issues and never being diagnosed with that to then later found out they had some form of it.
Thing is, even if this problem of yours has a name, there is no cure for these type of learning disabilities. In some universities you can ask for a bit more time and a calculator but that's it. I decided not to ask for those because 1. it was a long process 2. I was too proud (and stupid) to ask for it.
Honestly, my tricks were the following:
If it is a math test, I did thousands of exercises, even of the same type... I just did them over and over and over again. This does not mean that I did not do any silly mistakes (as you call them) but still, the chances were pretty low.
In case it was not a math test, I would still approach the problem similarly. Studying more (and I mean a LOT more) made me more confident and also less prone to silly mistakes.
I drank passiflora tea (I still do it sometimes) that made me more attentive during exams and helped me focus during long study sessions. Passiflora is also called the "Student's tea" because it has important effect on concentration and stress. I very much advice it to all students that have similar issues. Just do not drink three liters of that in one shot (you could hallucinate).
While all these made me pretty stressed at times, I always studied more than necessary and ended up with a PhD. I think that all depends on you. For some, it is natural to commit 0 silly mistakes, for me, this was a big challenge (as you said, at uni they really cared about certain details and I would get very little points at exams for similar errors). However, there is always a way to solve these problems, and you can definitively achieve what you want in life despite these issues it is all a matter of finding what works for you.
Hope this helped.