My opinion differs somewhat from the current answers (which primarily focuses on doing less and exploring more), which is why I'm writing an answer to perhaps balance things out a bit.
Some context. I worked pretty hard when I was 15 to end of 16 (I'm in Singapore which follows roughly the UK GCSE system so providing the specific age is probably more helpful here) in preparation for the O level national exams. That single exam determines your high school choices, which may affect your A level exams (at age 18), which affect your university choices, etc. So yeah, stakes were high. I was pushing myself to the point where I had my notes almost everywhere I go, to pull out and read whenever. This include family dinners, toilets, buses, you name it. I photocopied my notes so I can fold some and keep in my pockets at all times. That was more than 10 years ago, so looking back, these are my thoughts.
First, the pros. If you persist with this for long enough, everything else is going to feel like a breeze later on. You'll see people complaining about their long 9-6 hours and you'll go lol in your head. It translates into a strong work ethic, if only because you'll feel guilty about "wasting" time. You'll be really good at picking out lost time packets, those 5-15 minute chunks that people just dismiss. These obviously manifest itself into academic (and probably career) success later on, because what felt hard to others in terms of time devotion feels easy to you. A similar concept applies for jobs with long working hours (e.g. investment bankers, lawyers, doctors, when they're at the junior level), in that everything that comes after is going to feel trivial when compared to how hard they had to push themselves previously.
Second, early childhood academic success frequently leads to a higher self-esteem and greater confidence in what you can do and achieve. You're probably still going to feel self-conscious about your looks and stuff, but that positive feedback loop of working-hard=>achievement-praise-directly-related-to-hard-work (which apply in schools but may not in workplaces) is probably going to give you that assurance that so long as you work at it, you can achieve whatever you want. That you're good enough, and all that is lacking is work that you have the proven ability to commit to and deliver. This makes things like learning new hard subjects and skills easier and so likelier for you, which comes in handy. Career switches, esoteric subjects you need for your major/job, or just trying out skiing for the first time, it's fairly generic.
Third, and this is a point that may invite some disagreements. It's easier for you to commit to such a schedule now, when your life is simpler. I presume you don't have to worry about food, accommodation, you have splendid access to a pool of people from which it is very easy to make friends (school, common interests, stuck together during most of the day), and you kind-of only have one thing to do now to obtain "success", namely study. It's harder in the future when success is far more ambiguous and you find yourself having to perform well on several axes all at once, not all of which is evident to you until you learn it the hard way. So if you don't work hard now, it isn't going to be easier to work hard, or to figure out what to work hard on to get what you want later on.
So, yup. My advice would be this: Continue to do what you do. Never feel guilty about spending too much time working. (honestly, crucify me but I really don't agree otherwise. When you're 30 and neglecting your family and kids, sure. When you're 15 and healthy and your family members are healthy and there's a positive feedback effect in how success compounds itself in life? Err, no, clearly not.) Go all out for it. Put in your best shot. Exploit your potential to the max, never hold back. Know that what you have, is precious. Guard this motivation, defend it jealously, and exploit it fully. It will not last forever.
But.
Take note of the cons. Mitigate them somewhat, in an efficient manner. There's this maximisation-satisfaction idea, that you spend enough time and effort to satisfy some stuff, and then spend the remaining to maximise your performance in something else, in this case your work.
So, what to satisfy. This is obviously a non-exhaustive list, so be open to the fact that there will be others and look out for them, but otherwise here it goes.
First, your family. I don't have much information to work with, with regards to your family practices, so I'll just throw out a few. Try to spend time with them efficiently. Optimise for fully interactive activities that involve all, so you clear them all at once. Things like family dinners with everyone, somewhere nearby since the transport process isn't efficient for interactions. Talk to them more if you don't mind, your time is spent anyway so make the most out of it. Avoid things like long drives, movies, you know what I mean, the less efficient ones. Try to go for at least one proper dinner like this once a month. Or even family board games night, whatever works for your family. By the way, in case this is taken wrongly, I don't mean you spending this time as a checkbox thing. Genuinely interact with them. Have the mindset that you want to spend time with them, but time is limited so you'll make the best out of every second, instead of thinking of it as clearing a quota.
Second, your classmates and friends. Now, there are several time periods in school that is inefficient for studying, but optimal for such interactions. Things like recess, physical education classes, that sort. Make full use of them, drop those books then (maybe, decide for yourself which sessions are worth doing so or you want to do so) and commit fully to your friends and classmates. Again, see mindset from family, same thing here. Make friends, make new friends, but never feel obliged to go for stuff that you don't want. It's just going to breed resentment, you're going to hit the books harder because of the guilt and frustration, you'll be inefficient because of that, and yeah it just goes down from there. It's not worth it. You matter. Your choices, your decisions matter. Your time is yours, nobody has a right to them. Nobody. So, commit mostly only to those you want to go (but try to make your word count. Say no a lot right from the start, I know peer pressure, but trust me it's better than bailing later). I assure you, people care about your presence far less than you think. So, don't worry about it.
Third, physical health. Meh, you'll be fine, combine it with the friends time (or even family if they like that kind). PE lessons, go all out for it. Go for (some) badminton/soccer/whatever outings, you satisfy your friends/family along with physical health. Otherwise, it's mostly k, take care of your eyes, get a bigger monitor so you can increase font size/print your notes in larger font. People are going to ask why, they're going to say it's unnecessary, you're wasting paper/money. Whatever, ignore them, your eyes are worth way more. Distance matters a lot, try to increase it. Avoid strongly studying in low light. Also take note of your posture when studying.
Fourth, mental health. I split this from physical because it is important, even though you may not realise it now. You have to spend some time doing what you like. For yourself, not for others, not because it's efficient, not because it satisfies whatever. For yourself. Obviously don't take it to the extreme, but an hour a day on average is ok. On average, so if you're spending half a day gaming on a weekend, yeah that cuts into the rest. Two points here. First, this guy gets a free pass. If you need to extend, so be it. Console yourself by thinking that you'll be more efficient if you're motivated, and you'll be more effective. What's the difference? Being more efficient means you spend less 15-second blocks staring at the blank wall because you're bored. Being more effective means when you encounter a problem that you can solve but it reminds you you're not actually that familiar with a particular concept, you go to your book and read that section, rather than subconsciously dismissing it (you can solve it, right?). It matters quite a bit to your overall performance, more than what you might imagine, and happens more than you think. Be aggressive about filling in the gaps, but know when it's a fruitless time-sink and cut losses then.
Second point, see the above time spent on family/friends? Try to combine them, so you try to go for stuff that you genuinely enjoy, and avoid stuff that you don't. Think of it as a freebie. Viola, free time yo. Of course, if your family really wants to try that mexican restaurant but you hate mexican food, do it once in a while still.
Finally, explore other academic areas with a view on ensuring that you're not pigeon-holing yourself due to your interests. It's far too easy to say oh maths is really good as a college major/career choice, and incidentally I like maths. The former may be true, but you need to be aware that you're just rationalising here (but, the former may be true, so don't just dismiss). I'm not going to pretend that I know what the requirements for a top-tier data scientist 20 years down the road is going to need (although it sure as hell isn't geography or literature), so try to talk to some people who are in that field. Be aggressive about asking, put yourself out there, so long as you're genuine and nice and polite people will be ok with it. Send them a thank-you message/email after. Think of it as free advice, just grab it whenever you have the chance. Three points here. First, don't place too much weight on what your teachers say, they're detached from the job market, and many may never have even tried to get into industry, so they haven't gone through the what-are-they-looking-for search at all. Second, don't place too much weight on a singular person, no matter how esteemed. Obviously weight it by who they are, but don't place all your bets on a single horoscope reading (because predicting the future is). Third, trust yourself. If it smells like rubbish, it *probably is, even if that someone assure you it's not. Unless it's a few, in which case do some research on your own. Also, don't pigeon-hole yourself into single careers. Explore a bit, try more if it's fun, especially if it's assessed in your exams. Might as well.
That about sums it up from me. In direct response to your questions, yes studying for less than 4 hours isn't going to exclude you from whatever circles you wish to join. Still, if pretending that that is true helps in motivating you, go ahead. Some circles are quite competitive. No, I don't think you should drop any. If you like them, go for it. They're all really useful skills that scale very well. Number theory is highly abstract and that mathematical mental framework helps in making many college-level maths subjects easier to gain a deep understanding, I *think, depending on which level you're at. At a high school or US non-grad college level, yes it's quite transferable. Geometry scales well up to and incl college physics/maths, algebra scales to when you die, competitive programming and optimisation helps you understand OS/systems stuff at a deep level, machine learning is the new black (and everyone's wearing black, which is bloody annoying but whatever). By the way, and I don't mean to burn the US here, but you need to be aware that US standards for maths and related is kind of low. To put things in perspective, the singapore PSLE maths exam (which is taken at age 12) is comparable/harder than the SAT maths. Like, it's obvious it is, I'm not exaggerating. So, yeaaaaah. You can go a lot further, it's k, you're probably/definitely not so far ahead that you should stop.
Trust, but verify. You're a really mature and intelligent individual, your weakness is probably in experience. So trust yourself in the calls you make, but verify with some people whose opinion you trust. Some, not just one/a few, otherwise there's a tendency for selection bias. But make the call yourself, and bear the consequences yourself, you'll be fine. Dare to ask, many people (as you may have realised here from my long story) have too much time on their hands and are too eager to spew their hard-earned bullshit to anyone who would listen. So ask, accept first, and then verify, through personal research and perhaps asking others.
"I feel like if I study for less than 4 hours a day at least, I won’t be able to be a top tier data scientist or mathematician"part of the question. If no-one else gets to it first, I'll try to put one together when I have a little longer later...