If you did an opinion poll about opinion polls, chances are most people would recognise the limitations of market research, offer some unfavourable views of pollsters and deride the uses to which their work is sometimes put.
Yet if you asked politicians and the media whether polls deserve our attention, they would almost unanimously agree. Even after Brexit. Or Trump in 2016. Or the eye-popping poll earlier this month that found that one in five Brits support having a nationwide 10 p.m. curfew permanently in place, regardless of whether or not the pandemic is still raging.
Polls have major shortcomings. Even if pollsters avoid leading questions and interview the perfect cross-section of society, talk is still cheap. It’s a truism of economics that you cannot take at face value what people say. It’s why there’s a wealth of evidence showing that stated and revealed preferences rarely align.
For most goods, for instance, expenditure surveys and national accounts show broadly the same pattern. But less than half the recorded cigarette purchases show up in the Living Cost and Food survey. If we judged people’s desires by their behaviour — as economists do — we would not conclude that pristine health or self-improvement is their only goal. It would be reasonable to assume that many who expressed interest in learning a language or getting fit during lockdown are still sedate monolinguists.
While it can be useful to know which political party people are likely to vote for, new research from the Royal Society lays bare the pitfalls of using polls to show preferences on more complex issues. It calls for caution on how we interpret simple surveys, as there might be consequences to inferring that people either “support” or “oppose” a given measure.