Rory Sutherland

When more data makes you more wrong

From cricket to court, more information makes it easier to buttress a false narrative

When more data makes you more wrong
Text settings

In a one-day international against Australia last year, Ben Stokes was dismissed for ‘obstructing the field’, a rule rarely invoked in-cricket. The bowler had thrown the ball towards the wicket (and hence near Stokes’s head) in an attempt to run him out. Stokes raised his hand and deflected the ball. After some discussion between the two on-field umpires, and a referral to the third umpire, Stokes was given out.

What was most interesting was the difference in the conclusions people reached depending on whether they watched the replay in real time or in slow motion (you can find both on YouTube). Seen at speed, his raising of his hand looked nothing more than an involuntary and instinctive act of self-defence. Viewed in slow motion (as the third umpire saw it), it seemed a wilful and deliberate act of wicket--preservation. In the first instance, everything happened so quickly that it seemed impossible that Stokes had time to think; in slow motion it seemed impossible that he hadn’t.

But none of this matters compared with the effects when slow-motion replays are used in criminal courts. A recent University of-Chicago experiment showed subjects video of a bungled robbery in which a shopkeeper was shot dead: those ‘juries’ shown a slow-motion replay were more than three times as likely to convict as those who saw events at the correct speed. The effect diminished when the footage was shown at both speeds, but persisted: such juries were still one and a half times more disposed to find the gunman guilty.

Usually new technology works in the interests of justice. Advances in DNA fingerprinting have not only helped solve countless crimes but have exonerated thousands of wrongfully accused people worldwide. Such advances are also a powerful deterrent against committing crimes in the first place (especially for people like me who hate cleaning. I find it bad enough having to change a pillow case or duvet cover: faced with the prospect of replacing the carpets and swabbing the walls down with bleach, I’d probably just light a cigarette and wait for the police to arrive.)

There may be a small counter-trend in which technology — including CCTV coverage — makes false convictions more likely rather than less. Several pretty iffy judgments (Barry George, Knox and Sollecito, Siôn Jenkins) have relied on miniscule quantities of matter, meaning that the opinion of one vetted scientific expert can potentially condemn someone to jail for life. Mobile-phone evidence or CCTV footage — or the resources to analyse it — may be more widely available to the prosecution than to the defence.

Part of the problem lies simply in the greater volume of data now available. A fairly good rule of thumb is that the more data you have, the more gold is contained therein… but at the price of an even greater volume of false gold: spurious correlations, confounding variables and so forth. (Young people and university graduates were far more likely to vote Remain, but then young people are more likely to be graduates.)

Constructing an inaccurate but plausible narrative is much easier when you can cherry-pick from 50 pieces of information than from five. In one of the miscarriages of justice that Bob Woffinden analyses in The Nicholas Cases, great store was set by the fact that, on the fateful night, a wife had frequently called her husband but never the other way round; only later did it emerge that he had run out of credit on his phone. It’s a persistent problem with data. It doesn’t always make you right — it can serve only to make you more certain when you are wrong.

Rory Sutherland is vice-chairman of Ogilvy Group UK.