I will start in freshman in a fairly good undergrad institution in August 2018. I am planning to major in Mathematics. However this school attracts the most brilliant high schoolers in Math. This includes IMO gold and silver medalists. Participation in IMO guarantees admission to this school. I have already planned to go to graduate school. However seeing the enormous competition I am pretty skeptical of my chances. Usually the best student of our school in a given year has a fair shot at Princeton. Looking at my result I might have barely crept in their freshman batch. I do have 1 month left, shall I start on standard Math textbooks like Rudin, Munkres, Dummit and Foote or are the IMO medalists way ahead of me?
You are who you are. Don't be overly concerned about others. You will need to work very hard, of course, but the fact that others may be able to pick things up more quickly than you doesn't mean that you can't excel.
The fact that you were admitted suggests that others have faith in you. Have faith in yourself.
You can also try to find a study group with others at the new place so that you get additional exposure. Use good study techniques and never - never fall behind. Start early on assignments. Ask a lot of questions. I was once thought to be very smart because I asked a lot of questions in class. But in reality I just wanted answers that weren't apparent to me.
You might also try as much as possible to limit outside interests until you find your way in a new environment. However, agreeing with a comment by user aeismail, that doesn't mean to give up everything for academics. Spend several hours each week on something that contributes to your health, reduces stress, and engages your mind in a different way. I now use Taichi which does all of these things. As an undergraduate I practiced Judo several hours a week. I didn't get very good at it, but it was a great complement to studying. As a grad student I rode a bicycle with a local club - fast but not competitive. That was physically satisfying, but didn't have a mental element like Taichi does. But don't occupy yourself with trivia.
But yes, studying great books in advance ain't gonna hurt you any. I studied out of Rudin over 50 years ago and still have the book(s).
You might also find that some courses are more challenging than others Analysis v Algebra for example. That is natural as the core thinking processes and insights are just different enough that you can groove on some things and find others a bit more work. Put your best efforts where they do the most good.
Talk to your professors. Ask questions. Listen. Ask more questions. Correlation only matters for groups. You are who you are.
Why you ask questions:
Here is a story (real occurrence) from the University of Kansas about 50 or so years ago. A prof, quite well known and respected, was lecturing in Topology. This meant writing proofs on the board. Occasionally a step wouldn't be filled in with the statement "Of course it follows, trivially, that ...". On one such statement, a student didn't see the obvious connection and so asked. The prof looked at the board and the developing proof for a few minutes. Then walked over to the corner of the (chalk) board and started making notes to himself in a tiny script. He went into totally abstract thinking mode, ignoring the class. After a bit of writing and erasing, etc, he wandered out of the room toward his office. The students followed along, gathering outside the office. The prof started pulling books off of his shelf and consulting them - several books - several more minutes.
Then he seemed to find enlightenment and returned to the classroom. When everyone was again seated he announced. "Yes, of course. It's trivial."
In my own experience, I found that being successful at high school mathematics was detrimental to me. I went to a small rural school where math was a breeze and got to a large state school where I was placed right into advanced an mathematics course after doing well in a combination of the SAT Math and the University's placement test. I failed this course as I was unprepared for the necessary work and discipline needed to succeed. This was a wake up call that got me into gear, thankfully.
What I have found though, as I went through my graduate studies was that often, everyone in the program is good at math and is smart. Some are a bit quicker than others, but everyone is generally pretty bright. For the most part, the successful graduate students are the ones with the work effort and the grit.
If your read any of Julian Stanley's (and now Lubinski's and Benbow's) work out of Johns Hopkins on mathematical precociousness you will see that even the most brilliant mathematical minds can fail. Talent gives you a leg up on your competition, but being willing to work hard gets you across the finish line. And then being dedicated to your craft gets you ahead of your peers.
My point is, if you are willing to work, and put in the effort to maximize your abilities, you are already on the right track to being a successful graduate student and scholar in mathematics.
Also, one last bit of advice: If in class, you are not able to follow at first, you are unlikely the only one. Math is hard and complex and requires work to get good at. One of my favorite quotes is "...in mathematics you don't understand things. You just get used to them." ~Von Neumann (i.e., math is hard and requires work).
That said, relax. The admission committee isn't a charity organization. Unless (even if) you are from a minority/child of a rich alumni/famous etc, you wouldn't have gotten accepted if they didn't think you were capable of passing their curriculum.