Alex Massie

A Game of Numbers: Pollsters Go To War in Scotland

A Game of Numbers: Pollsters Go To War in Scotland
Text settings

On Sunday an opinion poll was just a poll. Nothing to be too excited by. Unreliable too. The real poll - the one that counts - is still a year away. So put not your faith in numbers. Disappointment that way lies.

On Monday the mood in the Scottish nationalist camp changed. Opinion polls now offered a persuasive and necessary reminder that Scotland's on the march. A march that ends in freedom and liberty and whisky next September as an ancient country reasserts her prerogatives  and takes here rightful place in the family of nations once again. Polls are pure dead brilliant, don't you know?

From which you will gather that two opinion polls have been released in Scotland these past few days and they cannot both be accurate. First a YouGov poll conducted for the (Unionist) DevoPlus campaign group hammered the Nats. According to Peter Kellner's outfit 59% of voters are minded to vote No and only 29% to vote in favour of independence.

Not so fast my friend! The very next day a Panelbase poll, commissioned by the SNP, arrived showing that, actually, 44% of Scots intend to vote Yes and only 43% to vote No. At last! A pollster brave enough to report what it really going on!

I exaggerate but only a little. On Sunday, after all, one detected a whiff of panic in nationalist ranks. There was much chuntering and whispering and people asking if YouGov could be trusted. Had they asked the same question as will be posed in the independence referendum itself? (Yes they had, albeit with a preamble about "leaving the United Kingdom" some nationalists consider "loaded"). Was their sample hopelessly loaded with Unionists and thus an impossible reflection of what Scots really think? I mean, something had to explain this otherwise inexplicable report.  Panelbase, of course, are beyond reproach. God's own pollsters.

Actually, it is possible that both these polls are outliers. YouGov may be under-estimating support for independence and Panelbase may be over-estimating that support. Even so, the disparity between their findings is striking. They can't both be right, can they?

Of course Panelbase might be right. The race may be tied at present. If it is then the Union is in real trouble. It might even be doomed. If the Yes vote is at 40% or more a year from the referendum then - from a punter's perspective -  betting on a Yes vote offers far more value than backing a No vote.

That's because getting to a Yes vote is a process not an event. It begins by turning No votes into Maybes, then shifting Maybes into the Yes camp. And this is the thing: once a voters has completed the journey from No to Yes how probable is it that, having made this mental leap, they will be minded to move back from Yes to No? Some may do so (a mind that has been changed once can be changed twice) but I'd be surprised if that proved typical. It is more likely, I think, that voters persuaded to vote Yes will find their commitment strengthening as the campaign progresses.

Motivation matters, of course, and Panelbase's polls are heavily weighted to those certain and all but certain to vote. (Nothing wrong with that.) Nevertheless, if I had to guess which of these duelling polls is likely to be more accurate I'd plump for YouGov's findings.

That is, in part, because Panelbase's questions also included these doozies:

Who do you trust to take the best decisions for Scotland: the Scottish government or the Westminster government? (Scottish government: 60%, Westmister 16%)

Do you agree or disagree with the following statement: 'Scotland could be a successful independent country'? (Yes, I agree: 52%. No, I disagree: 37%)

Those are respectable questions too (though I'd be worried, if I were in the Yes camp, by that 37% figure). But taken as a whole I don't think it's unreasonable to suspect that this is a poll nudging people towards affirming support for independence. Especially since these questions were asked before the main question on independence itself. That may not make this a junk poll but it's still enough to make you wonder about it. (For what it may be worth, John Curtice seems to agree with me. Or me with him.)

Panelbase polls have consistently found higher rates of support for independence. This does not make Panelbase wrong, merely an (extreme) outlier. Even so there are other reasons to question their findings. For instance, they report that 45% of women aged 35-54 support independence and only 41% oppose it. Given the longstanding and extremely well-documented gender gap on the independence question this is, to put it mildly, an eyebrow-raising finding even allowing for potential small sample size effects. After all, as recently as March this year another Panelbase poll found a mere 25% of women (all age groups) in favour of independence.

So, if pushed, I would guess that YouGov's poll is considerably more likely (though not certain) to be more accurate than Panelbase's. It is certainly more in line with the findings reported by other pollsters. It is Panelbase versus the field.

It still seems probable that support for independence is where it has always been: stuck in the low thirties and that, after a year of campaigning, the Yes camp has gained little ground. Perhaps the looming publication of the Scottish government's White Paper on independence will change that but at present neither side has conquered much new territory.

The difficulty for the nationalists and their colleagues in the Yes camp is that, historically, the higher the turnout the worse the SNP have fared. The SNP won 45% of the vote in the last Holyrood election but that was on just a 50% turn-out. They won only 22% of the vote at the last Westminster election when turnout was 63%.

Of course a single-issue referendum is not quite the same as either a Holyrood or Westminster election. Nevertheless it seems reasonable to presume that the 150,000 or so Tory voters who either did not vote in the Holyrood election or lent their support to the SNP will turn out to endorse the Union. Similarly there's little reason, I think, to suppose that Lib Dem deserters (who flocked to the SNP in 2011) have been persuaded to back independence.

Which means that the real battle, as ever, is for Labour voters. It is not, I think, a secret that Alistair Darling is concerned that a Tory recovery in England strong enough to make a second term for David Cameron look likely might persuade Scottish Labour supporters to vote for independence. Nor is it any kind of secret that Alex Salmond dearly desires a strong David Cameron.

Labour won 42% of the Scottish vote in 2010. That amounted to a million votes. Most polls suggest that perhaps 15% of those voters are currently open to voting Yes. I suspect, however, that the Yes camp needs much more than that to prevail. Indeed it probably needs more than 30% of Labour voters to vote Yes. That's not impossible though, again, it's tough sledding.

Realistically, however, how many more Yes votes are there? That is, how close have the nationalists come to maximising their vote? If you support independence there's been a party representing your views for eighty years. But that party has never won a majority of the vote, not even in low-turnout elections. To put it another way, if you wanted to win an independence referendum you would not choose to start from here.

Again, that doesn't mean Salmond and the Yes campaign cannot win, merely that the odds remain against them. And heavily so.