Skip to main content
[Edit removed during grace period]
Source Link
David
  • 8.4k
  • 20
  • 35
Source Link
David
  • 8.4k
  • 20
  • 35

One thing to bear in mind is that our ability to evaluate the solution to a problem is directly proportional to certainty with which we can answer questions about it. Very well defined problems can have very well defined answers. Less well defined questions necessarily have less well defined answers. The most well defined questions allow you to use numbers to solve them, but not all important questions fall into that category.

Social scientists aren't lazy, and there are many fields of social inquiry where statistical analysis doesn't really add to the conversation. To be clear, there are lazy scientists who could be doing things better, but you have to ask yourself what a detailed statistical analysis will contribute. We live in a culture that glorifies the scientific and assumes that if you can put a number to something then you know more than someone who can't, but that's not always the best way to approach things.

Consider an engineer who wants to determine the ultimate strength of a new type of metal alloy. They build a bunch of metal rods and stick it in a machine that pulls the rod apart until it breaks. They do this 30 times, recording the force required to break the rod each time, and then do some statistics to show that 90% of their rods broke within +/- 5% of 30,000 PSI. This is a very well defined problem that admits a very well defined quantitative solution.

Consider now an engineer who wants to build the strongest bridge possible subject to a $10 million dollar cost limit. This is a much more complicated problem, and given the huge number of design decisions it's not possible to actually find the single most optimal solution, but they can try many different possible designs and settle on the one that gives the highest strength while meeting all specified constraints. In this case the question is complex but admits a simple evaluation criteria (total strength), so it's easy to tell which of a hundred proposed designs is "correct"- the design with the highest strength.

Now consider an engineer tasked to build the most environmentally friendly bridge. Suddenly you don't even necessarily know how to formulate reasonable questions, much less find the "correct" solution. There is no well defined measure of "environmentally friendly". One bridge might generate a million tons of CO2 in the atmosphere to build, while another bridge might generate ten million tons of CO2. The second bridge runs through sensitive wetlands and will probably kill a species of endangered turtle. Is an extra 9 million tons of CO2 in the atmosphere worth killing and endangered turtle? We can quantify the possible alternatives precisely, but the question does not admit an optimal well-defined solution.

Likewise, psychology has some very well defined questions with very well defined answers. The military has done a lot of research into what it takes to train someone to shoot and kill another person on command. In WW2 a man named SLA Marshall did simple observational studies that showed that around 75% of troops would not shoot another human and did not fire their weapons in combat even if their own life was in danger. Thus, the psychological question "How many people are willing to kill another to save their own life?" is quantitatively and unambiguously answerable: about 25%.

There are more complex questions that have less well defined answers. The military's next question was how they could get more people to fire their weapon in combat. They experimented with a number of techniques, but in Dave Grossman's book On Killing he reports that by the time Vietnam rolled around the military was able to increase the firing rate from 25% to 90%. We can pose and answer a second question analogous to the engineer who wanted to build the strongest bridge: What training technique should the military use to maximize the firing rate of their soldiers?

However, Grossman and others have proposed that the increase in firing rate has increased the incidence of PTSD and related psychological disability by causing people to perform actions (with horrific consequences) that they are not naturally prepared to do. This raises a third question that is less well defined in the sense that the engineer's third question is not well defined. "How should we train our soldiers?" If you train them to be too aggressive then rates of PTSD and psychological trauma go up. If they're not aggressive enough then your soldiers die on the battlefield. Balancing those two concerns does not admit a simple measure that we can maximize or measure.

Difficult questions arise in every discipline- it just so happens that the natural sciences tend to ask questions that are clearer and unambiguous than the social sciences. That doesn't mean that the social sciences are less rigorous or less important, but it does mean that non-qualitative approaches are more prevalent in the social sciences.

http://www.gocomics.com/calvinandhobbes/1986/11/26 enter image description here