I am applying to PhD schools in physics in the United States. I would like to know if there is any advantage to applying early rather than just before the deadline. Honestly, I was trying to focus on research and I was also trying to write a good statement of purpose, hence this delay.
The only likely advantage to applying early is that you won't run into unexpected events that make it difficult to meet the deadline, and that this could improve the quality of your application (especially the written/essay portions) as well as avoid simply failing to meet the deadline at all.
That said, this can be a very valuable advantage. You'll find a lot of other questions here from people panicking about having missed application deadlines, having deadlines approaching too quickly, being unable to reach their writers for recommendation letters, etc.
I would not recommend applying extremely early (as-in, months in advance) - in that case, you'll instead be regretting having missed your most recent accomplishments. Give yourself some cushion against the deadline, and don't worry about gaming the system beyond that. No one cares (or probably even knows) if your application is officially submitted 0.01 or 1 or 2 or 3 weeks before the deadline: the deadline is there so that the admissions committee has a known day to start the job of reviewing while being fair to everyone. That's the only game.
Probably there is very little advantage. Committees are likely to only start considering candidates after the deadline anyway. Likewise not much disadvantage in being close to the deadline as long as you beat it. The committee will be made up (almost) entirely of faculty who want to schedule time and get through the pile of applications.
It is unlikely there is any advantage of applying too early, because unlike undergraduate admissions, there is usually no "quotas" to be filled here.
Contacting prospective supervisors early, however, is a very good idea. (1) They would give early feedback/pointers given your research statement, (2) They might give you advice if they're not themselves accepting new students (e.g. check out this recent paper and if you like considering contacting this or that supervisor).
And if you do contact them early, be sure not to make a commitment too early as well, as your research interest might shift after talking to a few professors.