Pollsters scramble to explain how polls could be so wrong in three provinces

OTTAWA – Canada’s pollsters have struck out and are now scrambling to explain how their predictions in three consecutive provincial elections turned out to be so wildly wrong.

Strike three came Tuesday when Christy Clark’s Liberals came back from the polling dead to easily recapture government in British Columbia.

The polls were similarly, though not quite as spectacularly, wrong in last September’s Quebec election and the April 2012 Alberta election.

Ipsos Reid, the country’s largest pollster, conducted an exit poll in B.C. that it says explains why its poll one day before Tuesday’s election — which had given Adrian Dix’s NDP an eight-point lead —was so far off the mark.

It suggests 11 per cent of B.C. voters made up their minds in the polling booths. That, combined with low voter turnout and a shift in voters’ priorities over the course of the campaign to the economy from wanting change, resulted in Clark’s stunning, come-from-behind victory, according to the Ipsos analysis of the exit poll.

“In this regard, our last (pre-election) poll should have had more attention paid to those who intend to get out and vote as opposed to just those who issued a voter preference,” the analysis says.

But Harris Decima chairman Allan Gregg said it’s just not plausible for pollsters to blame their failure to accurately predict results in three different provincial elections on a last-minute decision or shifts among voters to the winning party.

“The shortest answer is this should not happen, it literally should not happen,” Gregg said in an interview Wednesday.

“It’s the law of large numbers. If you do enough polling using properly framed samples and properly crafted questions, you will get the right answer. And if you’re not getting the right answer then obviously you’re doing something wrong.”

Gregg believes a “confluence” of factors are responsible for the problem: unreliable methods used to contact survey respondents, inexperienced, fly-by-night pollsters and badly-framed questions.

Traditional telephone surveys are no longer reliable since many people don’t answer the phone or have given up their land lines for cell phones. They tend to be skewed in favour of older voters who are most likely to support conservative parties, Gregg said.

Online polls — such as the B.C. polls done by Ipsos and Angus Reid — survey self-selected respondents who sign up to participate in a pollster’s Internet panel. Gregg said they tend to be skewed toward younger, urban, educated voters who are most likely to support the NDP.

As well, Gregg said a lot of inexperienced pollsters have jumped into the field, particularly in provincial campaigns, offering their surveys to the media for free as a way to promote their fledgling companies.

“Basically, charlatans who really don’t know what they’re doing or don’t have the resources or tools to do it properly.”

But even among the heavy-hitters in the polling industry, Gregg said there’s a misunderstanding of how campaigns work and what motivates people to vote.

Polls typically ask respondents which party they intend to vote for, which Gregg noted is not the question on the ballots voters actually cast. Ballots give the names of the local candidates running for election in each riding, which tends to give an advantage to well-known incumbents, especially in non-urban ridings.

“So what (polls) are doing is they’re wildly under-influencing the impact of incumbency,” said Gregg, who used to be the Conservative party pollster during the Brian Mulroney era.

That may explain why polls were so wrong in Alberta, where Premier Alison Redford was deemed a goner but pulled out a majority win, and in Quebec, where Jean Charest managed to come within a hair of hanging onto government despite polls predicting his Liberal party would be humiliated.

Sophisticated polls conducted for political parties plug in the names of local candidates when surveying voters, Gregg said, whereas polls conducted for the media — for free — generally can’t afford to go to such lengths.

Better internal party polling may explain why Clark confessed Wednesday that she “wasn’t as surprised as everybody else” by the B.C. result.

Ipsos senior vice-president John Wright disputed suggestions that there’s a problem with polling methodology. He said it’s just a reality that in some elections, such as B.C.’s, campaigns are hard-fought and voters “make up their minds in the voting booth.”

Clark appeared to agree with that assessment.

“The polls do not tell us how people are going to vote because voting day is the only day that they vote. It’s like me asking you what you’re going to have for dinner a month from now,” she said.

Still, Wright agreed with Gregg that the relatively recent influx of fly-by-night pollsters is helping to discredit the public opinion research industry and journalists’ addiction to polls has exacerbated the problem.

While he doesn’t favour banning polls during elections, he said: “We need to have a discussion about the use of polls in election campaigns.”

In Wright’s view, polls should not be published unless they’re accompanied by complete data tables detailing the response rate, the amount of weighting that was done to make the sample reflective of the population as a whole, and other factors.

“With no disclosure, there should be no exposure.”

Gregg suggested the country’s most credible pollsters should join forces during election campaigns to conduct credible national polls for the media on a not-for-profit basis, using a hybrid of land-line phone, cell phone and online surveys.

To allow the proliferation of unreliable campaign polls to continue unchecked “is completely dangerous” for the health of our democracy, he added.

“It absolutely changes the (media) coverage … and media coverage influences voting behaviour, there’s no question about that at all.”

Looking for more?

Get the Best of Maclean's sent straight to your inbox. Sign up for news, commentary and analysis.