Would it still be possible today to go from relatively unknown to a respected figure in physics by writing some spectacular papers being outside the academia? (Just like Einstein did)
My impression is that the likelihood of this is proportional to how theoretical the field that you have in mind is. In math or theoretical physics, it's at least imaginable for an outsider to produce an important new discovery. In High Energy Physics the idea alone seems outlandish.
That said, even in math the ratio of cranks (deluded amateurs who are convinced they have made an important discovery without having actually done so) to actual amateur geniuses is almost infinite to one. If you want to do research, in any field, the winning strategy is to study under a respected member of the community and learn how to produce, evaluate, and communicate your ideas in this way.
There is nothing that prevents you from doing so. It's just very hard for a couple of reasons:
- Without working at an academic institution (or something similar), you lack the environment to exchange and work on ideas with peers.
- Similarly, you need to discuss recently published results with peers to develop timely and relevant research directions.
- Every field of research has a certain publication culture. Without any advice on it, publishing is much more difficult. While you could get help from elsewhere, it's certainly a lot easier to get it from your advisor if you have one. If your results are obviously break-through and you know roughly how to write understandable scientific documents, this may not be so relevant, though.
- In case of experimental research, you often do not have access to the necessary laboratories and its equipment.
- Funding may also be an issue. In the experimental sciences, you may need to buy equipment or consumables for experiments. As another example, in computer science, you may want to publish at conferences, which comes with participation fees and travel costs.
- A good institution name can help to get you in contact with other researchers more easily.
- Access to publications is often easier as many universities have subscriptions.
There are probably many other reasons which I forgot.
I am not sure about the other fields, but with respect to Computer-Science/Engineering/Security-Research, you can easily do something on your own and become established. My personal path was somewhere along those lines. I dropped out of University to take a position doing research for the government full-time. It is worth noting that there seems to be a shortage in the field at large and so it isn't such a big deal to not finish college before you start working professionally. Not sure what the "ecosystem" of other fields of study are with regards to commercial vs academic.
Everyone's hedging, but essentially the message seems to be "not really." I say go for it. Research can help you find yourself, can sharpen your skills, and if it fails can lead you on new paths -- or if it succeeds can lead you to the next plateau, or to a resting place until you can figure out where to go next (if anywhere). Research is a luxury in that most people can't afford to take the time to do it. If you have the time and money, there's nothing stopping you. In fact, it's probably better than academia because there's no funding pressure or restrictions -- besides those you impose on yourself.
I think whether inside academia or out, research is a form of therapy. If you can stick to your objectives and not get sidetracked (or get sidetracked in a way that's constructive), I think you can discover much about the world. You shouldn't be looking for recognition, though. It should be about pure discovery. If you can get published, treat that as a bonus.
There's always the option of becoming a part of academia to further your goals. If you can afford to conduct research, you can afford tuition, and if you have the makings of a first-class scientist, you should have no problem getting into the college of your choice. I think this might be the most realistic option. Even if you're older, better late than never.
If you have money you can set up a foundation, to pay others to do the type of research that you think is important. You get name recognition that way. You can also donate to universities, provided they accept your patronage.
Yes, it is possible. There are enough people in the scientific community who actually value great ideas that, if you have one, won't care what your background is.
Of course, the idea would have to be new and useful, logically valid, and either theoretically compelling or backed up with plenty of evidence. There are a lot of skills required to do this with sufficient rigour. And it might take a lot of communication skills and perseverance to gain traction.
Would it still be possible today...
Have a look at The Structure of Scientific Revolutions. Science goes through phases, spending most of its time in "development by accumulation", but very occasionally being completely shaken by left-field ideas or solutions to confounding problems.
Development-by-accumulation typically needs very in-depth knowledge of a topic and often access to specialised equipment/resources/etc. But creative new ideas often don't rely on precise details of the old way of thinking. In fact, approaching things from an outside perspective can be an asset.
I don't agree with the "chances are tiny" way of thinking about this. If you selected someone at random then yes, the chances are negligible. But you're you, and need to be judged on your own drive and skills. Just don't have illusions. There is a reasonable chance you won't succeed.
One last thing—if you don't find that you frequently succeed at coming up with ideas about topics that you're not an expert in, you probably don't have what it takes.
I think it is possible just like it is possible to win the lottery. Unfortunately, the odds are so massively stacked against the possibility that it is unimaginable.
To author some "spectacular papers" whilst being outside of academia would require some knowledge of the field itself? Otherwise, how are you to know if it has been done or examined before?
Being frank, I would say this is more of a fairy-tale or a future movie rather than a reality.
Actually, it is much more suitable today for an "outsider" to contribute than before. Via the internet, the tools of information technology, the processing power of personal computing technologies, global and open access to scientific data and opinions, fairly inexpensive and reliable transportation. If you have the means to add time and finance to those, there is a real possibility of spectacular contribution in science.
That kind of achievement is easier/more probable in some particular fields like biology and geology. In these kind of fields, the size of the research area is huge and you can make invaluable contribution on a relatively small and less influential topic (small and less influential in regard to the whole field).
But that kind of achievement is less probable in some fields of science like physics, as you will need access to specific, sophisticated instruments and/or raw data. Even not impossible, it will be very hard.
But, necessity of instrumentation doesn't affect theoretical works and assessments on the data which are provided by other researchers. Because of the open nature and architecture (in general) of science, every researcher is like a colleague of others. As long as you can access necessary data and raw data when needed (obtaining raw data isn't very probable in most situations), you can do original research.
I feel a personal connection with your question, because of my similar thoughts and assessments on biology years ago. guess that either you don't want to be in academia for various reasons or you don't have the personal means (time, money) for this. I didn't want to be in academia in my country and hasn't finished master's degree program.
I think, for a special scientific contribution at last, you have to start with small steps and aim a research that will produce a synthesis of previous research (think a review paper). We can't ignore the possibility of a breakthrough research done by a researcher which isn't a part of the academic establishment.
It does happen albeit very rarely, and the relatively recent story of Lubos Motl (a noted contributor to Physics SE) comes close to what the OP describes.
If I became financially independent and wealthy (say by winning a few million pound in the lottery), and decided to spend my time learning physics and writing papers, even if I had considerably more talent than I have, my chances would be slim. You might look at a financially cheaper subject, like maths or computer science. The chances are somehow better there. And in these subjects, the limit is your imagination, not the real physical world around you.
But you asked “outside academia”: You may find a company that pays you well to develop products, and some might lead to papers that make you well respected, without being in academia.
And if you asked say the population of the U.K. for the name of a well respected astrophysicist, I think the name “Brian May” might be quite close to the top.
It does not matter how great the paper's discovery is if you are publishing regularly in important journals in your own field of research. The quality and number of revisions of your data is not as important as being published many times over in high quality peer-reviewed journals. Being published in quality journals automatically implies that some contribution has been made to the field.
Think about your aims before you start. Why do you want papers in the first place? Why not publish to trade magazines and businesses? Or where do you want to perform research? How would you measure success? You ought to think on that first. Having clear goals will make it apparent if your work will be viable without having access to academic resources. Academia and academic publications have become full of negative incentives, and not being part of it will make some goals difficult but not impossible.