Forecasts have been fundamental to mankind’s journey from a small tribe on the African savannah to a species that can sling objects across the solar system with extreme precision. In physics we have developed models that are extremely accurate across vastly different scales from the sub-atomic to the visible universe. In politics we have bumbled along making the same sort of errors repeatedly.
Until the 20th century, medicine was more like politics than physics. Its forecasts were often bogus and its record grim. In the 1920s, statisticians invaded medicine and devised randomised controlled trials. Doctors, hating the challenge to their prestige, resisted but lost. Evidence-based medicine became routine and saved millions of lives. A similar battle has begun in politics. The result could be even more dramatic.
In 1984, Philip Tetlock, a political scientist, did something new — he considered how to assess the accuracy of political forecasts in a scientific way. In politics, it is usually impossible to make progress because forecasts are so vague as to be useless. People don’t do what is normal in physics — use precise measurements — so nobody can make a scientific judgment in the future about whether, say, George Osborne or Ed Balls is ‘right’.
Tetlock established a precise measurement system to track political forecasts made by experts to gauge their accuracy. After 20 years he published the results. The average expert was no more accurate than the proverbial dart-throwing chimp on many questions. Few could beat simple rules like ‘always predict no change’.
Tetlock also found that a small fraction did significantly better than average. Why? The worst forecasters were those with great self-confidence who stuck to their big ideas (‘hedgehogs’). They were often worse than the dart-throwing chimp.
Comments
Join the debate for just £1 a month
Be part of the conversation with other Spectator readers by getting your first three months for £3.
UNLOCK ACCESS Just £1 a monthAlready a subscriber? Log in