Is it too old to start a Physics PhD at 26 ? In average , How many PhD students in physics or mathematics who are older than 25 in their first year ?
In the US at least starting at 26 would still put you in one of the largest age groups.
Looking at the data from the Council on Graduate Schools and the University of British Columbia, for example, one can see that there is a good range of Doctoral students in their 20s and 30s. In fact a number of other sources quote a median range of 27 to 34, for male, 32 for female - the last one seems to be the reason Google gives 33 as an answer if you search for "average age of a graduate student" in the US. So according to the data, you're actually a bit young.
As a personal example, I will be in my 30s when I start my Doctorate pursuit due to waiting until after my military service to focus on my academic career.