Pollster fears unreliable polls could trigger ban on surveys during elections - Macleans.ca

Pollster fears unreliable polls could trigger ban on surveys during elections


OTTAWA – A spate of spectacularly inaccurate polls in several provincial elections and Monday’s federal byelections has one of Canada’s leading pollsters worried that the day is fast approaching when public opinion surveys will be banned during campaigns.

John Wright, senior vice-president of Ipsos Global Public Affairs, said in his view shoddy polling methods by some companies and the media’s undiscerning appetite for horse race numbers — no matter how dubious — are dragging the industry’s reputation through the mud.

He wants the market research industry to crack down on pollsters who don’t meet minimum standards. And he wants media outlets to be far more selective about the polls they publish, rejecting surveys from companies that refuse to fully disclose all their weighted and unweighted data.

If that doesn’t happen soon, he predicted there’ll be a move to ban the publication of polls during campaigns.

“I think at some point, if this continues, there’ll be a real question raised whether these sort of things should even be contemplated being done during campaigns,” Wright said in an interview Tuesday.

Most polls were significantly off the mark in recent provincial elections in Quebec, Alberta and British Columbia.

Then on the eve of Monday’s four byelections, a Forum Research automated phone survey was published giving the Liberals in Manitoba’s Brandon-Souris a 29-point lead. In the end, the Conservatives eked out a win with 44.1 per cent of the vote to 42.7 for the Liberals.

The Winnipeg Free Press questioned the poll’s reliability, reporting that a number of constituents had been called as many as six times by Forum. It also carried the results, as did other media outlets.

If voters were in fact called repeatedly for the Forum poll, Wright said the survey sample would not be representative of the riding’s population.

In close contests like Brandon Souris, where less than 400 votes separated the Conservatives and Liberals, Wright said it’s especially important to ensure polls are accurate and properly conducted.

“If one vote was influenced by a bogus piece of work, that’s one vote too many,” he said.

Forum denies anyone was called more than once for the same survey.

In 1998, Wright said the Supreme Court ruled there’s no need to ban polls during campaigns because the industry and the media would police themselves sufficiently to ensure voters were not being misled by bad surveys.

“I think that’s a system that’s completely broken now,” he said, adding that he’s no longer sure the top court would rule the same way, given recent experience.

“Will there be banning of polls? Who knows,” Wright said.

“But certainly when you get enough of these things where you can’t tell whether there’s any merit or science or believability to it all and you get players who seem to disregard the very fundamentals of doing it the right way, then you wonder how far off it can really be.”

Forum’s Lorne Bozinoff defended his company’s record, noting that it accurately pegged the results in two other byelections Monday: Toronto Centre, Bourassa in Montreal.

“We know (automated phone polls) can work, there’s just no question about that. The question is why were they off in Brandon?” he said in an interview.

Bozinoff said “there’s no way” anyone was called more than once per survey, although some people may have been called for each of the three separate polls the company conducted on the Friday, Saturday and Sunday before the vote.

He speculated that the difference between the final Brandon poll and the actual byelection outcome may have been that the Conservatives had a better “get out the vote” ground game than the Liberals. As well, he said some constituents who were angry about the perception of a fixed Tory nomination may have found they just couldn’t bring themselves to vote Liberal once they got into the ballot box.

Whatever the reason, the Liberals’ supposed 29-point lead didn’t materialize — a point used by Conservatives to deflect attention from their party’s disappointing byelection results.

“The pundits were saying that we were some 30 points behind in Brandon-Souris … but our candidate did very well and we came out with a victory,” said Paul Calandra, parliamentary secretary to Prime Minister Stephen Harper.

The Tories hung on to Brandon and another Manitoba riding, Provencher — both long-time Conservative bastions. But their share of the vote plunged as much as 20 points in Manitoba and almost disappeared entirely in the Toronto and Montreal ridings.

The NDP gained strength slightly only in Toronto Centre, lost a bit of ground in Bourassa and sank into a distant third in the Manitoba ridings, after coming a respectable second in 2011.

The Liberal party alone increased its vote share in all four ridings, hanging on to the traditional Grit seats of Toronto Centre and Bourassa, despite an all-out challenge from the NDP, and making huge gains in the two Manitoba ridings.

Liberal Leader Justin Trudeau credited his positive approach to politics for his party’s momentum. But he faced criticism for going negative himself in his Bourassa victory speech, in which he used the words of late NDP leader Jack Layton to denounce what he called the negative campaign conducted by New Democrats in Toronto Centre and Bourassa.

“Make no mistake, the NDP is no longer the hopeful, optimistic party of Jack Layton. It is the negative, divisive party of Thomas Mulcair. It is the Liberal party tonight that proved hope is stronger than fear,” Trudeau said Monday night.

Mulcair retorted Tuesday: “That Justin Trudeau would use Jack Layton’s dying words as a political tool says everything that needs to be said about Justin Trudeau’s judgment and character.”

Layton’s widow, Olivia Chow, said she was “surprised and disappointed” to hear Trudeau appropriating her late husband’s words. While she insisted the NDP remains “the party of love, hope and optimism,” she said New Democrats should keep their sights trained on Harper’s Conservatives — which would seem at odds with the party’s targeting of the Liberals.

For his part, Trudeau was unapologetic for the Layton reference.

“The point that I made and I’m continuing to make is that this is no longer Jack Layton’s NDP. It’s very much Thomas Mulcair’s”


Pollster fears unreliable polls could trigger ban on surveys during elections

  1. Just looking at the last elections, it seems the pollsters have a built in Liberal bias. They cannot bring themselves to declare a Liberal as being behind. The last federal election had the Tories and Liberals neck and neck until the results came in. The Toronto mayoral election had Smitherman and Ford neck and neck until the results came in. Polls are no longer indicative of the reality on the ground but are being used as an advertising tool to generate interest in a candidate by convincing the public the candidate is a winner. Why run polls on Trudeau’s popularity when the election is two years away? They cannot possibly be getting accurate data as a number of people no longer have land lines.

    • this goes well for the tories in the next election, harper is going to use his experience and fiscal record to rip the left appart. this line says it all
      ‘The Winnipeg Free Press questioned the poll’s reliability, reporting that a number of constituents had been called as many as six times by Forum” DID SOMEONE SAY SKEWED???

    • The problem isn’t bias. The problem is the pollsters call only a segment of the population, and have no way to distinguish between someone who says they’ll vote a certain way, and someone who *actually votes*.

      With voter turnout as low as it is, polls are meaningless unless you can distinguish the voters from the non-voters. These things are constantly showing less support for the conservatives than they actually get on election day, because conservative supporters are more likely to show up and vote.

      Until they can fix that, the entire methadology is fatally flawed.

    • What you’ve claimed is inaccurate. Rob Ford was polling well ahead of his opposition months before the election. The Liberals were polling in third place in the federal election, not neck in neck with anybody; it was the NDP who were over-represented.
      The largest polling errors of recent elections were in the BC and Alberta provincial elections where Wild Rose and the BC NDP were overrepresented compared to the BC Liberals and Alberta Conservatives.

  2. It is long past time to ban polls during elections. Maybe the lack of ready-made stories will force reporters to write about issues and candidates for a change.

    • i agree. and not the candidates personal life

  3. No polls or surveys during campaigns….good idea.

  4. Doing polls on the cheap, small sample sizes, land line calls only, have you stopped beating your wife (yes or no) type questions and of course, telling the customer what they want to hear.
    Just another self-policing professional organization going down the drain. They can join the audit firms and the bond raters for consideration for an ignoble prize.

  5. How is putting out a fake, obviously unreliable poll, and then having multiple media outlets report it any less of a voter suppression technique than misleading robocalls? Are Forum research, the Winnipeg Free Press, Macleans, and any other media outlet that reported this ridiculous poll guilty of voter suppression?

    • Who did it suppress? I would think it would motivate conservative voters to get out and vote, and leave liberal voters comfortable with not voting because their candidate had it in the bag.

      • So you do believe that the Liberal vote was suppressed?

        Funny. I thought you’d be more upset given your candidate lost by only a couple of hundred votes. That voter suppression you are alleging might have cost the Liberals a seat, unlike any in the robocalls…um…attempted scandal. Damn conservative media…is there no end to their corruption? This time they actually did cause a seat to flip with their careless reporting of an obviously bad poll.

        So you’re going to pursue this? I think you should pursue this. You was robbed!

        • Well, all I was doing was questioning your assumptions. I know you are predisposed to see that everything that ever happens only happens because whoever it was who did whatever it was only did so because they are biased for the liberals. Just pointing out that this goes both ways.

          My belief is that the polls may not have been that far off, but that the CPC has more infrastructure in that riding, more volunteers and a better “get out the vote” team. That was the difference.

          But, unlike you, I do not thrive on anger. I may think a lot of things, but I am not going to run around railing against things every day. That would be kind of boring.

          • My belief is that the polls may not have been that far off


            Well, all I was doing was questioning your assumptions.

            No you weren’t. You said that the Forum poll may have convinced Liberal voters to stay home. “Comfortable not voting because their candidate had it in the bag”.

            That poll was so obviously skewed from a methodology perspective (people who vote Liberal being called 6 times!) that one has to question the motive of the group conducting the poll. Doesn’t seem like a random rogue poll (i.e. the “1 in 20”); this seemed like one with a purpose; to use a flawed methodology to present a picture other than the truth in order to affect voting intentions. And in a race separated by only 350 or so votes out of about 25K who voted? That’s a big deal.

            So I ask again…how is this poll, and any who reported it knowing that the methodology was flawed, any less guilty of “voter suppression” than the robocallers? They deliberately used a fradulent poll to convince enough people that voting wasn’t worth the effort. How is that any different than what robocalls was trying to achieve?

          • Are you suggesting the CPC commissioned the poll and paid for it to be skewed? Because that’s how it sounds from your comments.

            For what it’s worth, I do wonder what impact that skewed poll may have had on voting behaviour and whether that impact was enough to have cost the Libs the seat. Not sure there’s any reliable way to determine any impact though. It is, however, a good example of why I’ve been saying for years that polling ought to be banned during elections.

            EDIT: Just read the rest of your comments and from those it seems you think this was the Libs’ handiwork. Regardless of which party, it would be despicable if true.

          • IF anyone paid for it to be skewed (a very big if IMHO), I would suspect the Libs over the CPC. If the CPC were going to skew a poll to suppress votes, one would think from an optics perspective they would continue to make the seat look like a “safe” seat that Trudeau hadn’t made a dent in. But I have no evidence to suggest that the Libs did something like this; just suspicions.

            I’m starting to agree that polling during elections should be banned. It’s too corruptible.

          • Banning free speech is wrong. It doesn’t matter if it’s a pollster or a guy on the street corner.

          • Others have replied and made my case, but I will say a couple things.

            First, there are allegations of flawed methodology. The company refutes those claims. To date I am not aware of there is any proof of said flawed methodology (I heard someone say something happened, or someone said something happened is not proof – they are allegations).

            In any event, the get out the vote ground game was a big factor.

          • Calgary Grit has a pretty remarkable graph showing just how far off the bell curve this poll was. Draw your own conclusions as to how a poll could miss by this much. It’s pretty much a statistical impossibility for it to be random chance.

            He also comments on GOTV. I agree completely with his comments…GOTV can make a difference of a few percent. Not 30.


          • One more thing

            It is obvious that the conservatives own polling showed they were in trouble.

            It is not that I disagree that a deliberately skewed poll is wrong – I am questioning your assumption that this was deliberately skewed, and if so it was done to benefit the liberals.

    • Well, the robocall case (if it’s as it’s alleged) is a deliberate effort to mislead. The bad poll is irresponsible to be sure, but it’s not a deliberate effort to mislead. Further, the Conservative Party was DIRECTLY involved in the robocall affair (if it’s as it’s alleged). The Liberal Party was not directly involved in this crappy poll.

      You’ll also have a tougher time proving that the bad poll suppressed any one of the parties. It could have lit a fire under the Conservative supporters. It could have lit a fire under the Liberal supporters. It could have suggested to Liberal supporters not to bother voting, they had it in the bag. It could have suggested to Conservative supporters not to bother voting, they were gonna lose anyway. Whatever argument you want to make about the bad poll benefitting the Liberals, I can easily give you an equally compelling argument that the poll benefitted the Conservatives.

      I’m sure it could have suppressed a vote, but I’m not sure which party would have benefitted from it. You’d have a tough time proving a case either way.

      • The bad poll is irresponsible to be sure, but it’s not a
        deliberate effort to mislead.

        Are you sure? Have you read about the problems with the methodology? The complaints about Liberal supporters being polled 6 times? That suggests the possibility of a fraudulent or manipulated poll, not just a “bad” (i.e. the infamous 1 in 20) poll.

        I’m sure it could have suppressed a vote, but I’m not sure which party would have benefitted from it.

        At the end of the day, does it matter? A vote suppressed through false pretenses is a vote suppressed.

        And FWIW, I don’t agree with you or Gayle. People like to vote for a winner. If there was intent to deceive here, it was to discourage CPC supporters from coming out in a “meaningless” byelection to support an obviously doomed campaign. These effects are more pronounced in a byelection as well, where there is already only a fraction of the usual turnout for a general election.

        And one more data point to consider. Forum apparently does polling for political clients. Darrell Bricker is aghast that a polling firm, and the media who report those polls, are not disclosing that relationship. Care to hazard a guess who Forum’s political client(s) are/is?

        • I can tell you with all honesty, when I read the poll I thought, first, ‘that can’t be right’, and second, ‘this will help the Conservatives.’ Maybe my instincts are wrong, and maybe instincts aren’t the same as other folks, but… those were my first gut reactions.

          • I could buy that with a poll showing the Conservatives down 3. Maybe down 5. Beyond that, not credible.

          • Well, I certainly found down 29 unbelievable, but I racked that up to incompetence rather than design.

            FWIW, I don’t like any poll, even ‘accurate’ ones. You should base your vote based on who you think would be best at representing you — not on how everybody else is going to vote. I think polls have potential to skew voting behaviour across the board. And, I do agree with you that if polls are being used to directly influence voting behaviour, it isn’t right.

          • I could support a ban on polls during elections or something like that — but I also have to say that every place I saw that reported that bogus Forum poll that said LPC was so far ahead also said that Forum was unreliable. Perhaps media outlets should have skipped reporting on it, but you know how it goes — if one does, they all have to have something to say about it. I do think that polls released on or around election days can impact voter behaviour, but it is less clear that it has the same impact on all voters — so such news could well have ensured CPC supporters got out the vote. It seems that it would be more likely to influence CPC supporters to vote, and to influence LPC supporters to not bother.

  6. Reason why Polls are not accurate:


    Most media is bias toward the left and they generally hate conservatives so polls used in news stories support left wing politics.

    The left leaning reporters try to sway elections.
    As per training by left wing teachers from kindergarten upward.

    Trudeau senior was a big believer in an elite ruling class moderating the excesses of the artisans. (workers)

    • LOL.. your complete lack of any factual basis for your claims is hilarious. The only way media *can afford* to be biased toward the left is if it’s not advertising supported. Any media that relies on advertising for existence will be forced to move bias toward the right and a pro-corporatist agenda, for the simple reason that media groups which attack their corporate advertisers run out of money to do so. The only way media can be considered biased toward the left is if your personal centre lines up with Bill O’Reilly.

      As for those left leaning reporters you talk about, just about *every* media group last election supported the CPC. A few individual reporters did not, but even then only because of the behaviour of this particular government, over the last period, not because of any preference toward the policies of more “leftward” parties.

      As for your liberal education.. if that was actually the case, there wouldn’t be a conservative left in the world, now would there?

    • Thwim’s comments are pretty much on the mark. My significant other is currently doing a paper on media bias for a Poli Sci course and I’ve been reading some of the materials. The media is without doubt to the left of most of the CPC supporters who post on here decrying media bias; it tends to be centre-right, balancing the advertisers’ interests with that of their readership (two thirds of Canadians are not CPC supporters, so swinging too far right loses them audience share). Further, there is evidence that the media is slowly moving farther right – including (and especially) the supposedly leftist CBC.

      It made for interesting reading; it mostly supported my own take on things, but the bit about CBC moving right surprised me.

  7. I never thought I’d find myself in even minor agreement with Justine (Son of Margaret), but when he says the NDP is no longer Jack Layton’s NDP, I agree with him.

    But I’m also thankful.

    Mulcair is one of these marxist wackos saying, “Let’s put a government bought chicken in every pot.”

    Jack on the other hand only liked young chicken and would have loved to see a government bought massage table at every Bixi Bike location.

    • Troll

      • Well at one time I thought the party of “love and hope and optimism” was a good thing.

        But when they became the hypocrite party of “love and hope and optimism and statues of Rubber Jack,” I changed my mind.

  8. Unsurprisingly, the pollsters have excuses. They always have excuses.

    The fact of the matter is that polling no longer works reliably. The combination of the decline of land lines and decreasing response rates to polls means they’re reaching an ever shrinking part of the population. Among that group, they have no way of knowing if they’re reaching someone who will actually vote or not, as turnout is decreasing. They always wind up underestimating Conservative support because of the turnout issue: Conservative voters are more likely to show up on voting day.

    There’s been too many failures in too short a time for any excuses to hold water. The methodology no longer works. A ban is a great idea. As a bonus it’d force the media to do some reporting instead of yammering on about the horse race numbers.

    • they have no way of knowing if they’re reaching someone who will actually vote or not

      Or if those they reach are answering honestly. Very important with the shrinking sample sizes.

  9. Banning free speech is wrong, period. If a pollster is no good, then don’t pay attention. This is supposed to be a free country.