X

Create an account to continue reading.

Registered readers have access to our blogs and a limited number of magazine articles
For unlimited access to The Spectator, subscribe below

Registered readers have access to our blogs and a limited number of magazine articles

Sign in to continue

Already have an account?

What's my subscriber number?

Subscribe now from £1 a week

Online

Unlimited access to The Spectator including the full archive from 1828

Print

Weekly delivery of the magazine

App

Phone & tablet edition of the magazine

Spectator Club

Subscriber-only offers, events and discounts
 
View subscription offers

Already a subscriber?

or

Subscribe now for unlimited access

ALL FROM JUST £1 A WEEK

View subscription offers

Thank you for creating your account – To update your details click here to manage your account

Thank you for creating your account – To update your details click here to manage your account

Thank you for creating an account – Your subscriber number was not recognised though. To link your subscription visit the My Account page

Thank you for creating your account – To update your details click here to manage your account

X

Login

Don't have an account? Sign up
X

Subscription expired

Your subscription has expired. Please go to My Account to renew it or view subscription offers.

X

Forgot Password

Please check your email

If the email address you entered is associated with a web account on our system, you will receive an email from us with instructions for resetting your password.

If you don't receive this email, please check your junk mail folder.

X

It's time to subscribe.

You've read all your free Spectator magazine articles for this month.

Subscribe now for unlimited access – from just £1 a week

You've read all your free Spectator magazine articles for this month.

Subscribe now for unlimited access

Online

Unlimited access to The Spectator including the full archive from 1828

Print

Weekly delivery of the magazine

App

Phone & tablet edition of the magazine

Spectator Club

Subscriber-only offers, events and discounts
X

Sign up

What's my subscriber number? Already have an account?

Thank you for creating your account – To update your details click here to manage your account

Thank you for creating your account – To update your details click here to manage your account

Thank you for creating an account – Your subscriber number was not recognised though. To link your subscription visit the My Account page

Thank you for creating your account – To update your details click here to manage your account

X

Your subscriber number is the 8 digit number printed above your name on the address sheet sent with your magazine each week. If you receive it, you’ll also find your subscriber number at the top of our weekly highlights email.

Entering your subscriber number will enable full access to all magazine articles on the site.

If you cannot find your subscriber number then please contact us on customerhelp@subscriptions.spectator.co.uk or call 0330 333 0050. If you’ve only just subscribed, you may not yet have been issued with a subscriber number. In this case you can use the temporary web ID number, included in your email order confirmation.

You can create an account in the meantime and link your subscription at a later time. Simply visit the My Account page, enter your subscriber number in the relevant field and click 'submit changes'.

If you have any difficulties creating an account or logging in please take a look at our FAQs page.

Books

The prophetic fallacy

13 October 2012

9:00 AM

13 October 2012

9:00 AM

The Signal and the Noise Nate Silver

Allen Lane, pp.534, £25

This book isn’t just about prediction, or even the limits of knowledge. It is about the ascent of man. According to Nate Silver, the American electoral analyst, the digital age and its explosion of knowledge constitute a great turning point in human history. Never before have we had so much evidence on which to base our predictions of the future. Yes, there have been setbacks along the way, but we should feel optimistic about the direction of travel.

Yet there is a paradox about the evolution of prediction. Innovations in technology and information make it possible to analyse data far more quickly and extensively.  The availability of so much raw data, however, means that much of the analysis is woeful. There are many more bullets; but also many more inept marksmen. T. S. Eliot’s question has never been more apt: ‘Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?’

The Signal and the Noise acknowledges this, but Silver remains an optimist, hoping to build on the tradition of Enlightenment self-confidence. A similar case for optimism was memorably expressed 15 years ago by Peter L. Bernstein, whose Against the Gods argued that modern finance constituted the front line of the great march of knowledge over uncertainty. The revolutionary idea that defines the boundary between modern time and the past is the mastery of risk, argued Bernstein. ‘Until human beings discovered a way across that boundary, the future was the murky domain of oracles and soothsayers who held a monopoly over knowledge of future events.’ Unlike Bernstein, Silver has to temper his optimism to account for a massive financial crisis — which the combined predictions of regulators, ratings agencies and experts failed to spot.

This book ranges confidently across sport, politics, weather and gambling. Perhaps the less you know about the subject, the more there is to enjoy. I was riveted by the chapter on chess. Until recently, the best chess champions were able to defeat even the most powerful chess computer. That changed in 1997 when ‘Deep Blue’ beat a tetchy Gary Kasparov. Ironically, Kasparov was not defeated by a masterpiece of computer code. He got spooked by a bug, which he misinterpreted as higher intelligence — and ended up chasing shadows, identifying depths of strategic insight that weren’t always present.


Silver emerges as a modest, engaging narrator and an admirably clear thinker. But he might have explored more deeply why we find it so hard to disentangle two separate threads when thinking about the future. The first is prediction: what will happen? The second is strategy: how can I capitalise on what happens? Silver’s book assumes, not untypically, that the latter relies on the former — that a good strategy must be predictive.

But there is an alternative view. Rather than hitching ourselves to the game of prediction, what if we develop strategies that reduce our exposure to uncertainty by being more sceptical about the future — and hence more flexible and adaptive? The best strategist I’ve met — he was professional sportsman before becoming an educationalist — disliked the term strategy and was impatient with prediction. Instead of wasting time trying to predict, he focused on being open to change — hence benefitting from the light-footedness of not being attached to a predictive view of the world. His strategy, in effect, was anti-strategy. By decoupling the arts of prediction and strategy, he embodied what Nassim Taleb, author of The Black Swan, calls ‘antifragility’ (the subject of Taleb’s new book).

This approach is often misrepresented — usually by people who are scared of uncertainty or have an ‘evidence-driven’ spreadsheet to deliver to their boss — as pessimistic and unpractical. In fact, antifragility is sceptical but not remotely fatalistic.

There is a further problem with the thirst for statistically-driven prediction. It can encourage the mistake of turning all disciplines into pseudo-science. Journalism is a case in point. There is nothing innately ‘rigorous’ about over-liberally sprinkling dubious findings from soft social sciences on top of every argument. Some insights can stand on their own two feet.

Perhaps that is another way of saying there are many different kinds of truth, that science does not have a monopoly on describing reality. Which more truthfully captures human nature: Anna Karenina or the latest dodgy experiment that claims to have discovered the ‘neurological causes’of love (‘his pre-frontal cortex was firing off so much oxytycin that he jumped off his horse and offered his hand in marriage’)?

The ability to make data-derived predictions belongs to a much broader intellectual challenge: distinguishing which subjects can be addressed scientifically and which cannot. As a practitioner, Silver scored hits by making predictions about baseball and US elections. As he points out, both disciplines lend themselves to statistical analysis. But the approaches that work for baseball and elections are not universal. Summoned to speak at a conference about terrorism, Silver can only agree with the critic who replies, ‘That’s all very well, Nate. But how in the hell is this applicable to terrorism?’

So does the genuine advancement of certain forms of prediction outweigh the harm done by overconfident predictions in spheres that do not lend themselves to statistical analysis? Silver says it does. Even those who are less convinced will enjoy his balanced, intelligent and erudite advocacy.

Subscribe to The Spectator today for a quality of argument not found in any other publication. Get more Spectator for less – just £12 for 12 issues.


Show comments
Close