*After I published this article, I got some feedback from a political betting website contributor pointing that YouGov were actively sifting out the politically over engaged from their sample. As it turned out, Labour outperformed expectations at the 2017 GE, and YouGov's seat model, much mocked in the media, was spectacularly accurate.

The Problem with Opinion Polls?-Polls?-Polls?-Polls?-Polls?...


In the lead up to last year’s EU referendum something very significant, and very sad, happened; Leave were flying in the opinion polls and as short in the betting as they had been all campaign, then, on June 16th, Labour MP and Remain supporter Jo Cox was murdered by a man who would almost certainly have been a leaver. The political analysts agreed it was a blow for the anti EU movement, when polling resumed there was a marked shift towards Remain, and Leave drifted from 2.4 to 15 on the betting exchanges. A week after Cox's death, Leave won 52/48; there had been no shift to Remain.

Before GE 2015 the polls had the Conservatives and Labour vying for the lead. The narrative amongst betting experts was that No Overall Majority was "free money", even at prohibitive odds, and the media dismissed talk of either party winning a majority, busying themselves with the permutations of the incoming coalition. At 10pm on election night, the exit poll had the Conservatives as the largest party by some margin, and early next morning it was confirmed that David Cameron's Blues had racked up 331 seats, commanding a majority in the House. The coalition calculations were rendered redundant, the NOM money was lost. Ed Miliband never had a hope.

The current opinion polls show Jeremy Corbyn's Labour Party cutting Theresa May's Tories lead to shreds since her manifesto was launched on May 18th. At the start of the month the Conservatives had three leads of 20% or more; since the manifesto was unveiled, and its controversial social care policy revealed, there have been several in single figures. The talking heads have revised their predictions of a three figure majority downwards, the Corbynista's (and Blairites) are smelling blood. So it seems obvious - the voters have read what the Conservatives have to offer, and don't like what they see... this could be close!! Diane Abbott could be Home Secretary!!!

 If the polls called it incorrectly in the last two national elections, why should we take seriously what they tell us now? Why did they indicate one thing, pretty strongly, only for the public to say something different? I believe it’s because they are not recording the opinion of the public as a whole but extrapolating the opinion of the type who like answering opinion polls - the politically engaged. The resultant swings make good copy for the hacks, and fuel suppertime conversations of the chattering classes... but are they all nonsense?

"It's a concerning trend that polls now often driving the news - and becoming political events in themselves. Same true in 2015." - @Ed Balls 30.05.17

The politically engaged are a tiny, but enthusiastic, percentage of the population. The difficulty for opinion pollsters is that they are not representative of the public. They follow politicians and political journalists on twitter, they post about politics on forums, they watch Newsnight and the Daily Politics, digest the info and answer polls. They like to show off their understanding and want everyone/anyone to know that A GREAT DEAL OF THOUGHT HAS GONE INTO THIS. As most men in the pub discuss football, they are online discussing politics. While the man in the pub will generally quite bluntly say who he (always) votes for when asked, the politically engaged find such partisan loyalty an affront to critical thinking - being seen to be "undecided" is a badge of honour, it shows they are a serious person. They admire intellectual reasoning and put a high price on their vote, so when the chance comes to answer questions on how they think and why, it's like giving someone a big line of cocaine and asking them to talk about themselves. Political obsessives are the material of opinion polls, but not the fabric of the nation. It could be that in showing off about doing their homework, giving the "clever" answer rather than what they actually intend to do, they are making the polls less accurate.

An example of the politically engaged's answers being beneficial was YouGov's polling for the Labour leadership contests, and I feel this backs up my theory... the material of the polls WERE the voters in those elections, and the polling was very accurate as a result. But in General Elections and referendums, each vote carries the same weight regardless of how invested the voter is. So we have swingy polls, heightened media interest in the polls, frenzied analysis of the media interest and the polls.. when the result was the same all along - The opinion polls, coverage of opinion polls, and reaction to coverage of opinion polls feed off each other to create an unrepresentative echo chamber.  

How can pollsters improve their results? One possibility could be to ask each respondent how politically engaged they are, how often they watch QT, Newsnight, News at Ten etc and weight their responses accordingly, skewed heavily towards the politically disinterested. Maybe they should exclude the politically engaged from political opinion polls altogether.

It may not be popular with the chatterati, but their considered view is considerably less important than that of the bloke down the pub.