This raises several difficult discussions. The TL,DR is: Don't worry.
First, one thing is to talk about reproducibility when studying physics or chemistry. If I can make electrons eject from a metal sheet by some method, so can you, as long as you "reproduce" my metal. In this case, both the matal, a source of light and some electron detection device should be accessible for another research group without much difficulty. And usually there is an underlying mathematical foundation that allows one to work around difficulties in the reproduction. I.e. if you don't have the same metal or the same light source, maybe you can figure out a different material and light frequency that works for what you get. Are you reproducing the my exact claims? No. Are we jointly validating (or attempting to falsify) the same general thesis? Yes.
If however, I'm trying to make a medicine study, and my sample is the whole population of New Zealand. How can you expect to reproduce my results? Will you test the same people again? Will you try to test the whole population of Equador or some other country? Or maybe a whole province in China? If you get discrepant results, will you claim my results are false?
If you like to think like a mathematician, Imagine I give you a black box that presents a different number every time you touch it. And I give someone else a black box that looks exactly the same. For some reason, both of you believe that the sequence of numbers they generate is very important, and you start conducting research on it. With enough research, you are going to conjecture and decide that it is impossible to falsify that both black boxes are embedded with a random number generator, but each one has been initialized with a different seed. You'll convince everyone of this thesis if you are both able to record all generated numbers, figure out the random number generator algorithm and the seed of each box. To make things more complicated, imagine the generated numbers are actually feed to some kind of Infinite Impulse Response filter. And that the coefficients for each box are different, and also need to be discovered for the scientific truth of the box to be uncovered.
Now, try to imagine that while you are touching the box, recording numbers and researching its behavior, you also need to publish results.
First, raw data is hard to publish. You can put it on a public database these days, but it does not constitute an actual publication. Remember: publish or perish. Even if you do, your date is not the same os the other box. If a researcher on the other box publishes his data, that is widely different than yours, would you claim that he is reproducing his experiments?
Now, as part of scientific investigation, you will try to figure out facts about the numbers. Are they correlated? Is a moving average nearly constant over a wide enough window? Is there a trendline? But the numbers are random, so what if none of this properties are matching? Do you actually expect that reproducibility will be possible when studying such machines?
And see that there is indeed a well defined and deterministic behavior for this machine, but it depends on parameters that are unknown, and are not replicated in the different boxes as they're found. Yet, nobody knows this for sure as the search goes on. People might start to conjecture that the numbers are absolutely random, that they are affected by room temperature, time, weather, vibrations and so on. Some people will claim that there is absolutely no logic to these numbers, and that the boxes are completely different.
Given this scenario, try to answer your own question. One day, someone might be able to discover the algorithm, filter structure and a method to find the parameters and seed. This would mean that every box owner can reproduce the method, the results, and start forecasting the future outputs of the box. Until them, nobody has found the truth, and even incomplete parts of the discovery would be very hard to prove by themselves.
But so what? Should people not publish anything? Do we consider this box to be pseudo-science and not fund it? (To be clear, the hypothesis is that everyone is convinced those boxes are important, though if we actually performed this experiment, they wouldn't be).
But this enters another question: How do you measure return on investment on scientific research? And should some return be expected at all?
If PhD students pay nothing for the classes they're taking, equipment they're using and still receive some stipend. I'm pretty sure they have to deliver some result. Otherwise, we'd be talking about social welfare, not about research funding.
Notice that there are companies, trusts, funds and other profit oriented organizations that will promptly provide money for a high enough chance of return above the risk-free rate. Then again, I've rarely heard of this financial sponsoring within academic environments. There are cases were companies fund a few scholarships mostly to ensure that enough people will be trained on a certain field of knowledge, where "acquiring existing knowledge" is a goal different than that of "performing research", which is "developing novel knowledge".
People often justify that spin-offs or unprecedented applications justify research which initially is pure science (rather than applied science). Every time someone says this to me, I ask them if they've ever developed or at least read a business plan. Usual answer is "what is that?".
As far as I've been told, no university earns money with patents. Not even MIT, or Florida University (which I heard was the highest patent-earning university in the US).
There is the case where after 20 years of research, only a few more two are needed before a commercial product is developed with the technology. This a matter of science maturing into technologies, and while people love to tell stories of unexpected success, there is a survivor bias applied there, as nobody enjoys hearing about failures.
Then again, a few fiels do have expected sigma-levels, quoting this article:
" For publication in Physical Review Letters, for instance, 5-sigma results are generally called 'observations' or 'discoveries,'" (...) 3-sigma results usually warrant calling the result "evidence" of a new particle, but scientists need more statistical certainty to include terms like "discovery" in a technical paper.
As you may see, this kind of requirement, which falls into the lines you've probably expected as an answer is specific to a journal. I doubt it is field-wide standardized.
So, if you are the editor of a well-respected journal, you can impose your classification system (what constitutes evidence vs. discovery). And you can choose to accept or reject papers based on how convincing the evidence with respect to how feasible it is to obtain evidence at all. But don't expect everyone too agree with you.
Also, note that there is this underlying belief that observations and/or physical phenomena have gaussian-distributed fluctuations. Which may or may not be the case for the boxes example. For arbitrary distributions, a bound on the probability of a value deviating so many times the standard deviation can be given by Chebyshev's Inequality, which provides a much more loose bound. Even 5 sigma for Chebyshev means only 96% chance. But hey... Where did sigma even came from? Did you estimate it? From the data? What is the certainty you have on estimating standard deviation?
Also, I've mentioned survivor bias, there is the obligatory mention of XKCD's Green Jelly Beans Discovery.
To make my point clear, I think a lot of research should focus developing methods, which in turn may find their validity by usage, or how often then can find information that resist falsification with available data and experiments. In turn, many methods create new kinds of questions, rather than answering previously conceived questions.
However, your question seems based on the mindset of answering laymen questions "with science", which I believe is problematic in itself.
For starters, science requires precise language, which laymen don't speak, laymen (and myself on my daily life) ask simple, but likely ambiguous questions whose answers they might not understand. If I ask: "Can every even integer greater than 2 be expressed as the sum of two primes?", I'm asking a simple question, but note that "prime" in this sense is a technical term. You have to study a bit before understanding it. And If someone asks me this question, I can randomly answer "yes" or "no". But that is is not "answering with science" (though, granted, I'm talking mathematics). Despite considerable effort, this question remains an open problem. So if it ever gets solved, I would expect years of training for the answer to be understood.
To give a contrasting example: If I ask "Is cabbage good or bad for my health?". How do you measure good and bad? Those are un-precise terms. And also, I've asked "for my health". So, we may simply not have tools to infer that whatever finding you had for a test sample is applicable to me. Even then, if you manage to "test goodness of cabbage" for a subject's "health", as long as there is a single discrepant result, you should not extend your answer to me. And notice: I want a "yes" or "no" answer. I don't want you asking me about my colesterol, allergies, blood type, hours of sleep, other eating habits, type of cabbage, weather and common diseases on my city. I may not even know how to answer these questions.
But, if you could isolate all these variables, run a DNA test on me, maybe you could identify certain diseases I'm less prone to develop if eating cabbage regularly. That is insightful, that is scientific. And it requires you to collect much more samples to determine all variables that need to be isolated. Notice that those variables are part of studies that may reach "inconclusive results" by some sigma-level metric. And this goes back to my argument, that data and methods need to be developed, even if no conclusion is in sight.
Finally, we have the problem with corrupted/falsified data/observations due to researcher's mistakes and or poor ethics. Honest mistakes are usually random and uncorrelated (though there are questions about common misconceptions in several fields spread out SE sites). They are the main reason some people argue there is a need to reproduce results.
But fake data of fake is a different problem. Deliberately fake data or results are no different than a deliberate false claim. However, is it really a relevant issue? A good journal should demand reproducibility means from its potential publishers, so fake data may at best waste the editor's money investigating false claims. Assuming that fake or irreproducible research does not find it's way to top journals, and if it does, it should be relevant enough to draw enough attention of people equipped to debunk it in a few years.
Do you actually trust sketchy journals and small conferences? Anymore than laymen news sources? And here is the laymen problem again. He reads news outlets. He expects them to translate complex "scientific discoveries" to his language. He doesn't even ask for the original sources and proper citations. He might as well believe a hoax shared on social media as much as he believes an actual scientist. But this a problem with basic education, not with science itself. Implementing scientific conclusions on daily life is also not a straightforward process. That is, there is a long way between researches finding out that "cabbage is good for health" and nutritionists actually start recommending cabbage for patients. There is a reason why one should prefer talking to a specialist despite the fact that information is widely accessible. There is a big gap between having information and knowing what to do with it.
But what if... These false claims become medical products? That is why more than scientific journals, we need regulation agencies. They are the guardians of whatever methods for reproducibility or further proof should be required before alleged discovers become accessible for a broader audience.
After all this discussion, has your question been answered? I'm guessing you think "no". But then again, aren't you trying to ask a scientific question with a laymen mindset here? Even if there was a methodology to assert a sigma-level required in each field before publishing conclusions, would it be reasonable to systematically reject publications based on this criterion? I'm inclined to say no.
