Tell me, do you want to be in academia or not? If not, then leave academia even if you are the most suited personality for it. If yes, then stay in academia even if people like you are not welcome. When the place for people like you isn't carved out in academia, cut one out for yourself. Follow your curiosity. Curiosity has its own reason for existingTM.
Your weakness is your strength too. There is a positive term to describe that "stubbornness" of yours - Academic Guts - which means going into a problem and not giving up on it.
To be honest, I'm not entirely sure how I'm still in grad school.
We deserve the place we are at, whether moon or gutter. You are in grad school because you earned it.
I've been researching under one professor's wing for two years now, ostensibly shooting for a Ph.D. (note: no topic yet), and I have accomplished nothing except write a lot of code, fill a bunch of notebooks with theorems and scratchwork, and whine a lot.
Writing a lot of code is something. Just in case you like writing code, think about becoming a Software Engineer or a Computer Scientist. "Filling notebooks with theorem" is how an outsider would describe your work. I, too, have filled a lot of notebooks (even though I'm a Software Engineer), and my manager never liked it in the first two years. I paid heavily for that - denied promotion and on the brink of getting fired. But after two years, finally, the results came, and those filled notebooks helped design the best possible solution to the team's problems... Solutions that were good enough, that when my manager formed a new sub-team to solve a critical issue, the sub-team was composed of only Senior Software Engineers plus me.
You can stop whining - it never helps.
Why are you shooting for Ph.D.? What do you want to achieve through Ph.D.? Do you know what you want to do? If you don't have an answer, give yourself some time and think about it.
Insofar as I can surmise, my precise set of skills is good for one thing and one thing only: Solving problems that have already been solved... but only from scratch, my way. And I'm too stubborn to know when to give up.
You concluded that your skills are suitable for solving already solved problems because you re-solved the research papers. In other words - you like attacking problem statements. Always those problem statements come from research papers in your case. Go to a professor and ask him what problem (s)he is working on. That way, you will get a problem statement to attack, and another researcher would not have already solved the problem. Working on an unsolved academic problem sounds like Ph.D. to me.
I can't read academic papers in my field. [Either] I can't read it because I'll get too easily excited [Or] I can't read it because it means nothing to me [and] my mind races about other things as I'm reading.
It is a sign that you don't have a clear goal. You probably need to focus your field further down. When reading papers, your mind races everywhere because you have not fixed the purpose. Fixing purpose is more important than improving focus. If your goal is fixed, where can your focus go?
Stop reading just any research paper. Fix your domain. Analyze yourself and see what kind of problems you like to solve. See what specific fields you like in Physics. If you like too many, pick one and do well in it rather than scattering the power of your mind all over the place.
I want to share an anecdote with you from my own life:
I could never fix my field of study in life (even when I was done with my undergrad). I always thought that if I kept doing something in field X and never touched field Y, I would miss all the fun in field Y. With this, all my life until undergrad, I neither thoroughly enjoyed Xs nor Ys. I was like an oscillating pendulum - suffering initially with severe mood swings and later with severe depression. I lost my friends, and they often left me with suggestions of seeing a psychiatrist.
Totally fed up with my life, I took a year off after my undergrad. During that year, I went through a lot of self-help material, and whenever I had any insight into my life, I wrote it down in a diary. However, the turning point came when I met one particular farmer.
This unsuspecting farmer had the answer to my every single question (Not kidding at all - He discussed Math, Physics, Politics, Life, Death and even told me the limit of the universe!). Regarding my indecisions, he said to me that when all his friends chose to become industrialists, politicians, engineers, and whatnot, he left his home searching for peace and studied life. He then decided that he would continue with the profession of his ancestors - farming. Now, after almost 37 years since he chose to be a farmer and live a happy and prosperous life, those friends of his often come to his place and in the evenings relax in his fields, saying - "We have been fooled. You are the only one who is actually living the content life."
The simple point - Do one thing and do it well. Even if you want to become a good farmer, you will need to be educated about almost everything. Pick one field for yourself, and soon you will realize that to be good in your area, you will eventually need to know everything because everything is connected. However, when your field is fixed, your awareness of everything will expand in a controlled manner, unlike how it is happening with you now when your mind just races.
If I have to show it with a drawing, below is the depiction of a mind whose purpose is not fixed. This mind jumps all over the place - races faster than light - and doctors call it ADHD†. (The line shows the location of your thought.)
Below is the pictorial representation of another mind. This mind has its purpose fixed. This mind commits no mistake because it is entirely focused on the task. This state of mind is also called Meditation - all the attention focused on the one point.
...ultimately, there's a point where everything goes into a black box, and the result pops out. This is a shame because I know that these computations can expose new and interesting emergent properties of the theory. Still, sometimes it's just too difficult finding the ladder down from my ivory tower.
You are right - there are exciting things to be discovered in those "black boxes." I can understand you when you say that it is just too complicated. The fact is, everything worth doing takes effort. Do you want to stay on the surface and do simple things? Or do you want to delve deep and do difficult things that can cause paradigm shifts?
Even better - how about not worrying about the ease or difficulty of the problem and just solving them because you enjoy solving such riddles? That is the only way forward where you will be happy. Let me tell you another story -
In my grad school, I decided that I would master "Data." During that time, the word "Data" meant nothing more than "Databases." Everyone, including professors, TAs, and classmates, told me that there is no sense in studying Databases as it is a very static field. There isn't much growth and not many avenues to explore. It was upsetting and slightly discouraging that the area I chose for my study after the year-long break after undergrad was not considered good by people.
Little did everyone know what was going to happen. I invested my entire Master's in studying the ins and outs of Data. I took every course with the word "Data" in its title - even against everyone's recommendation. When I was just about done with my Masters and beginning to look for jobs, thanks to Google's work and white papers, this new term called "Big Data" was hugely hyped, and industry was abuzz with it. One of the top three jobs at the time of my graduation was Data Scientist. The same set of people who disagreed with my choice earlier later told me that this field has a lot of scope.
Is there really any place in academia for somebody incapable of recognizing and building upon the existing work of others?
The answer is simple and has already been given at the top - If you want to do something, do it. Honest efforts never go to waste. Hard work will surely pay. Don't be tied to the materialistic dividends of hard work. The immediate result of honesty and hard work is peace and satisfaction. Good things take time. Your habit of doing things by yourself will show its effect in some time. After hundreds of self-solved research papers, for example, you will get so familiar with the proofs that you will not just read the equations; instead, you will visualize them. That will be a gift unique to you. Who knows what beauty will come out of that Pandora's box?
Update (Dec 3, 2022): I came across this beautiful article today that explains my point from another perspective. People complain of attention deficit. This person was hyper attentive.
†Some readers have an objection to me stating that the behavior of a person with an unfocused mind, with their thoughts jumping all over the place, is (the root of) ADHD. I have been provided links to three research papers that seem to substantiate the same. Below are specific sentences from those papers:
1. Attention-Deficit/Hyperactivity Disorder is not a single pathophysiological entity and appears to have a complex etiology. There are multiple genetic and environmental risk factors with [a] small individual effect that act in concert to create a spectrum of neurobiological liability.
2. The structural aspects include changes primarily affecting but not limited to the prefrontal cortex, corpus striatum, and cerebellum [...] Larger studies should be conducted to have a better and definitive understanding of the pathophysiology of the disease.
3. At this stage, no definitive conclusions can be offered regarding the utility of meditation-based interventions for children with ADHD and/or their parents, since the methodological quality of the studies reviewed is low.
In response to the linked papers, I have the following points to make:
1. It is pretty obvious to me that our actions are a direct result of our thoughts. You can evaluate the same yourself. Ex: A person who has chosen to live kindly and makes conscious efforts to be kind in thinking, is incapable of performing a cruel action. Simply because the basis of being cruel is missing in them.
2. Not a single one, out of the three papers given the link to, makes a clear or authoritative statement that the brain's structure is the primary cause of ADHD. I think it is completely fair to expect that if the understanding of ADHD, or any disease for that matter, is clear, then the paper would have strong statements about the same, rather than soft statements like the three stated above.
3. 'Fuzzy logic' is no logic. When there is logic, there is no fuzziness. And even further, correlation is not causation. Some humans may be prone to exhibit hyperactivity, just like some people have long strides because they have longer legs. But, in the absence of a clear demarcation of the physical and conscious aspects, the solution of a non-physical "disease" is impossible. And to be even more clear, 'attention' or its deficiency is conscious/non-physical; 'activity' or hyper-ness of activity, is physical.
4. I wish to emphasize once again - Meditation is not a treatment of ADHD. Meditation is the opposite of attention deficiency. The root of the definition of meditation is - 'Focus one's mind for a period of time.' and that is the literal opposite of attention deficiency.