Jonathan Jones

Why British GDP figures are almost ALWAYS wrong.

Why British GDP figures are almost ALWAYS wrong.
Text settings

Will it be 0.5 per cent? 0.8 per cent? 1 per cent? Whatever figure the ONS tells us GDP grew by in the third quarter of 2012, there's one thing you can be pretty sure of: it won't be the actual amount GDP grew by in Q3. In the past 51 years, just 12 of the ONS's 205 first stabs at quarterly growth have survived later revisions. To be fair, the ONS recognises this, and cautiously labels tomorrow's figure a ‘preliminary estimate’. But just how wrong is it likely to be? If tomorrow's figure is +0.5 per cent, does that mean we can be pretty confident that growth was between, say, 0.3 per cent and 0.7 per cent — bad but not disastrous? It turns out we can't. In fact, a preliminary estimate of +0.5 per cent really means somewhere between +1.2 per cent and –0.2 per cent — anything from encouraging to dire.

Since 1961, the ONS's first estimates of quarterly GDP growth have been revised by an average of 0.8 percentage points. Here are those revisions for each of the 205 quarters from 1961 Q2 to 2012 Q2:

And here's that presented another way, with the height of each bar reflecting the frequency of that particular size of revision:

As you can see, it's not unheard of for the initial estimates to be out by more than three points. In June 1971, for example, the ONS estimated that GDP had fallen by 4.5 per cent in the first quarter of the year — by far the worst drop in any three-month period. The ONS now says that GDP fell by just 0.9 per cent — bad, certainly, but nowhere near as terrible. Or take 1988 Q3: the first estimate of –2.3 per cent has since been revised to +1.4 per cent.

But there is an ONS success story here too. As the top graph shows, preliminary estimates have become a lot more accurate. There hasn't been a miss of more than 1.3 points since 1988. And in the last 20 years, the average revision has been 0.4 points compared to the one-point average in the previous 30 years.

From 1961 to 1992, the margin of error for first estimates was +/– 2.2 points. That is to say, 90 per cent of the final figures were within 2.2 points of the preliminary estimates. So if the first estimate was +1 per cent we could only say with 90 per cent certainty that growth was between –1.2 per cent and +3.2 per cent.

Since 1992, though, that margin of error has been much smaller, but still +/– 0.7 points. These two graphs, showing the distributions of revisions in the two periods, demonstrate how much the first estimates have improved:

As I say, the ONS does, to its credit, recognise the uncertainty in its estimates. They point out that there's an inevitable trade-off between timeliness and accuracy. Timely statistics, they say ‘are needed by users so that their decisions better reflect current economic circumstances’. And they proudly boast that, coming just 25 days after the quarter has ended, the ONS provides ‘the fastest official estimate of its type produced by any major industrialised country’. In 2004, Marian Bell, then a member of the Bank of England's Monetary Policy Committee agreed that ‘we would prefer early imperfect data to late perfect series — it gives us something to work with’.

Fair enough then. The ONS puts out an initial estimate that may well be several tenths of a percentage point out, because it is better than nothing, and considerably better than a guess. But you'll have to comb through tomorrow's release carefully to find a mention of the average size of revisions (last quarter it was at the bottom of page three, filed under ‘Supplementary analysis’). And you'll find no mention at all of the margin of error or confidence interval — a crucial part of any estimate. Instead, the first page will tell you that ‘GDP increased by X per cent in Q3 2012’. Really, it should say ‘GDP increased by X per cent in Q3 2012, plus or minus 0.7 points’. That might muddy the headlines a bit, but it would at least be more honest.