Short: There is a progression in rigor and abstraction that goes like this: High school physics, high school math, university engineering, university physics, university math. There are large differences between different universities and between different schools.
Assuming that an Austrian Matura is similar to a German Abitur or Swiss Matur (I don't know for sure), then even courses for physicists and engineers are going to have more rigorous math than you are used to, but it's a natural progression and if you have a passion for the subject and liked math ok in high school, then you should be fine.
Courses for mathematicians (which you might be required to take if you study physics) are at a level of rigor and abstraction that is not comparable to the math required for a Matura. If you think about enrolling in a program that requires such courses, I suggest you look through some exercise materials for linear algebra I and analysis I or whatever your study program of interest requires - try to solve some and skim the solutions for the rest. You will get a much better idea of what it would be like to study those subjects for real than by reading abstract descriptions of what it was like for other people.
That being said, some schools offer much more preparation than others. There are Gymnasien which offer an optional math course which is actual math, statement, prove, example, repeat. Some offer more hands-on math that will make the transition to an engineering or physics program easier, but won't make much of a difference if you decide on math. Some might offer additional engineering courses, etc.
The universities in Germany are going to assume the least amount of previous knowledge that an Abiturient would have (which varies a lot among Bundeslaender), but starting university life can be a stressful transition and if you have some more than the minimum it's going to make the first weeks or even months a lot easier.