Rogers Media uses cookies for personalization, to customize its online advertisements, and for other purposes. Learn more or change your cookie preferences. Rogers Media supports the Digital Advertising Alliance principles. By continuing to use our service, you agree to our use of cookies.
We use cookies (why?) You can change cookie preferences. Continued site use signifies consent.
Someone I know on the Hill has been agitating for this to be flagged. I post it now both to bring an end to his incessant emails and to explain why I am otherwise hesitant to draw your attention to the latest poll numbers.
Quebec voters, meanwhile, appear to be abandoning the Bloc since the last election while the Greens have seen a huge surge in the province (difference in brackets):
Bloc Quebecois: 22 per cent (-16)
Liberals: 24 per cent (0)
Conservatives: 17 per cent (-5)
NDP: 12 per cent (0)
Green Party: 26 per cent (22 per cent)
I think the poll is an outlier as well. What have the Greens done in the past few months that would account for their lead in the polls. Unless Quebecers are so sick of all the major parties after the Coalition fiasco they have decided a pox on all their houses, why the high Green numbers?
For a simplistic question involving hypothetical voting intention, you don’t need a huge sample. So long as the sample accounts for sufficient representation (gender, income, etc.) and is truly randomized, a few hundred individuals per province (in this case) is often adequate (again, for a very simple question of this nature). The Quebec numbers have a 6.3% error potential, which is on the high side, but that’s not strictly a function of sample size (i.e., you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work.).
Plus, this could be the dreaded “20th” case (19 times out of twenty being the standard confidence), where all bets are off.
The Quebec numbers have a 6.3% error potential, which is on the high side, but that’s not strictly a function of sample size (i.e., you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work.).
This is completely wrong. The margin of error is ENTIRELY a function of sample size. The bigger the sample size, the smaller the MOE.
“The margin of error is ENTIRELY a function of sample size.”
I may have overstated things a bit, but MOE is not entirely a function of sample size. Confidence, sampling technique, population size, and the spread of resulting data all can affect error. While it is true that larger sample sizes lessen the MOE, issues of sampling and the distribution of responses can influence it, as well as playing with confidence level. Population is less of an issue, and I don’t see polling firms tweaking the confidence level that often (though I notice the article does not state the confidence for this particular poll).
Mathematically, MOE is a function of sample size and population size. Look it up if you don’t believe me. Or calculate it: http://americanresearchgroup.com/moe.html
The other stuff you mentioned is relevant to polling in general, but not to the calculation of MOE.
Your statement that “you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work” is completely wrong.
In other words, you made a mistake. Be a man and admit it.
Crit_Reasoning on
The term margin of error strictly applies to only random sampling error: in practice, it is very commonly conflated with different sorts of sampling error.
I see no reason for a pedantic tone.
Quite obviously, if you polled 10,000 Anglophones people in Montreal, you’d have a pretty severe sampling error, if, strictly speaking, an extremely low MOE in the “proper” sense. If you both went on a news show to talk about the 10,000 Anglo poll, and he said it had a high MOE and you said it had an extremely low one, you’d be laughed out of the studio too quickly to explain what you meant. With finger raised skywards and a look of triumph, no doubt.
Jason Townsend on
JT, the poll in question was a random telephone poll, just like all their other polls. So other sources of sampling error are highly unlikely, which is why I didn’t address them. Most likely, the wonkiness of this poll is some kind of human error (like data entry) and has nothing to do with statistical errors at all.
Crit_Reasoning on
I agree entirely on the cause of the error; CG puts it succinctly near the bottom of the thread.
I was just defending Sean Stokholm. He should have said sampling errors rather than margin of error, and the point doesn’t explain this poll. Sampling errors are, however, important, a point that entered our political discourse in a big way during the great poll-watching season of ’08, with everyone debating the number of minorities R2K was polling and the daily posting of running polls and other such minutiae which go far beyond just looking at sample size and random sampling error.
The post made at 2:09 was entirely the wrong tone and factually wrong. I’m sorry to belabour it and that isn’t your tone now, but that’s why I replied. These are illuminating discussions we can have without urging one another to be less stupid or unmanly.
Jason Townsend on
That still only makes a margin of error in the 6.5% range. Not enough to account for the huge jump.
Clearly a wacky poll. It might be accurate, but I don’t think anyone could rely on it until another poll confirms it. Until then, assume it is the 1 in 20 that is totally off.
Well Wherry alludes to a hesitancy to even publish it but doesn’t explain his misgivings. Is it due to the wonkiness of teh result? And are the final numbers the only wonky thing about it, or are there other reasons to not want to post it?
Look at the Green party numbers. (26%) This is OBVIOUSLY a mistake by someone at CTV or Strategic Counsel. Looks like a case of someone typing the wrong numbers into the wrong column – some kind of human data entry error. Nothing to do with statistics. I expect a correction will be issued soon. Nothing to see here folks; move on.
Given that a lot of other polls recently have had the Bloc at around 40% and the Greens around 5% the fact that anyone could consider these results accurate is laughable. Clearly this is a case of the 1 time out of 20 where the results are completely wrong.
Not to go all math-professor on everyone here, but there’s no such thing as a “1 in 20” time when the results are insane. It’s gradual. The 6.3% moe means that we’re 95% sure the results for the Greens are between 20% and 32%. But, we can also say that we’re 99% sure they’re between 18% and 34%. Or that we’re 99.8% sure they’re between 16% and 36%. So, based on these numbers, there’s a 1 in 500 chance the Greens have at least 16% which is still an insane claim to make.
And the margin of error is conservative in nature because it takes the safe assumption that the probability is 50%. If we say the Greens are really at 10% (which is a stretch in and of itself), the moe at 95% confidence is actually only 3.8%. (the principle being, it’s hard to be REALLY off when there are less people who fit into the group. So you’ll never see the Marxist-Lenninist party at 6%, even if the moe is 6%, because there are so few of them.)
Given that the Greens got 4% in Quebec last election and there’s no reason to explain a surge in support for them there, it’s clear this poll is completely off and it’s not just a “randomness” thing.
Anyways, the mid-term is next week. Be sure to read chapter 6 on Bayesian statistics by then…
I’m sure this will help SC’s reputation. Considering the Green Party didn’t rush to throw this up on their site should be a pretty good indication as to the reliability of this poll.
I noticed that at Kinsella’s place. Greens leading in Quebec makes me question the poll, personally. Perhaps bad sampling…
Andrew (not Potter or Coyne) on
I think the poll is an outlier as well. What have the Greens done in the past few months that would account for their lead in the polls. Unless Quebecers are so sick of all the major parties after the Coalition fiasco they have decided a pox on all their houses, why the high Green numbers?
jwl on
What e-mail address is Elizabeth using now?
Dot on
You might want to point out that the Quebec sample was only 244…
Érik on
For a simplistic question involving hypothetical voting intention, you don’t need a huge sample. So long as the sample accounts for sufficient representation (gender, income, etc.) and is truly randomized, a few hundred individuals per province (in this case) is often adequate (again, for a very simple question of this nature). The Quebec numbers have a 6.3% error potential, which is on the high side, but that’s not strictly a function of sample size (i.e., you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work.).
Plus, this could be the dreaded “20th” case (19 times out of twenty being the standard confidence), where all bets are off.
Sean Stokholm on
The Quebec numbers have a 6.3% error potential, which is on the high side, but that’s not strictly a function of sample size (i.e., you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work.).
This is completely wrong. The margin of error is ENTIRELY a function of sample size. The bigger the sample size, the smaller the MOE.
Crit_Reasoning on
“The margin of error is ENTIRELY a function of sample size.”
I may have overstated things a bit, but MOE is not entirely a function of sample size. Confidence, sampling technique, population size, and the spread of resulting data all can affect error. While it is true that larger sample sizes lessen the MOE, issues of sampling and the distribution of responses can influence it, as well as playing with confidence level. Population is less of an issue, and I don’t see polling firms tweaking the confidence level that often (though I notice the article does not state the confidence for this particular poll).
Sean Stokholm on
Mathematically, MOE is a function of sample size and population size. Look it up if you don’t believe me. Or calculate it: http://americanresearchgroup.com/moe.html
The other stuff you mentioned is relevant to polling in general, but not to the calculation of MOE.
Your statement that “you could have a sample of 10,000 in Quebec, and still have a high +/- factor at work” is completely wrong.
In other words, you made a mistake. Be a man and admit it.
Crit_Reasoning on
The term margin of error strictly applies to only random sampling error: in practice, it is very commonly conflated with different sorts of sampling error.
I see no reason for a pedantic tone.
Quite obviously, if you polled 10,000 Anglophones people in Montreal, you’d have a pretty severe sampling error, if, strictly speaking, an extremely low MOE in the “proper” sense. If you both went on a news show to talk about the 10,000 Anglo poll, and he said it had a high MOE and you said it had an extremely low one, you’d be laughed out of the studio too quickly to explain what you meant. With finger raised skywards and a look of triumph, no doubt.
Jason Townsend on
JT, the poll in question was a random telephone poll, just like all their other polls. So other sources of sampling error are highly unlikely, which is why I didn’t address them. Most likely, the wonkiness of this poll is some kind of human error (like data entry) and has nothing to do with statistical errors at all.
Crit_Reasoning on
I agree entirely on the cause of the error; CG puts it succinctly near the bottom of the thread.
I was just defending Sean Stokholm. He should have said sampling errors rather than margin of error, and the point doesn’t explain this poll. Sampling errors are, however, important, a point that entered our political discourse in a big way during the great poll-watching season of ’08, with everyone debating the number of minorities R2K was polling and the daily posting of running polls and other such minutiae which go far beyond just looking at sample size and random sampling error.
The post made at 2:09 was entirely the wrong tone and factually wrong. I’m sorry to belabour it and that isn’t your tone now, but that’s why I replied. These are illuminating discussions we can have without urging one another to be less stupid or unmanly.
Jason Townsend on
That still only makes a margin of error in the 6.5% range. Not enough to account for the huge jump.
Andrew (not Potter or Coyne) on
Clearly a wacky poll. It might be accurate, but I don’t think anyone could rely on it until another poll confirms it. Until then, assume it is the 1 in 20 that is totally off.
Charles on
Well Wherry alludes to a hesitancy to even publish it but doesn’t explain his misgivings. Is it due to the wonkiness of teh result? And are the final numbers the only wonky thing about it, or are there other reasons to not want to post it?
Michael (someone else) on
Look at the Green party numbers. (26%) This is OBVIOUSLY a mistake by someone at CTV or Strategic Counsel. Looks like a case of someone typing the wrong numbers into the wrong column – some kind of human data entry error. Nothing to do with statistics. I expect a correction will be issued soon. Nothing to see here folks; move on.
Crit_Reasoning on
Given that a lot of other polls recently have had the Bloc at around 40% and the Greens around 5% the fact that anyone could consider these results accurate is laughable. Clearly this is a case of the 1 time out of 20 where the results are completely wrong.
Miranda on
Fantastic numbers as far as I am concerned and I hope they hold true. Then again I’m a conservative.
Wayne on
Not to go all math-professor on everyone here, but there’s no such thing as a “1 in 20” time when the results are insane. It’s gradual. The 6.3% moe means that we’re 95% sure the results for the Greens are between 20% and 32%. But, we can also say that we’re 99% sure they’re between 18% and 34%. Or that we’re 99.8% sure they’re between 16% and 36%. So, based on these numbers, there’s a 1 in 500 chance the Greens have at least 16% which is still an insane claim to make.
And the margin of error is conservative in nature because it takes the safe assumption that the probability is 50%. If we say the Greens are really at 10% (which is a stretch in and of itself), the moe at 95% confidence is actually only 3.8%. (the principle being, it’s hard to be REALLY off when there are less people who fit into the group. So you’ll never see the Marxist-Lenninist party at 6%, even if the moe is 6%, because there are so few of them.)
Given that the Greens got 4% in Quebec last election and there’s no reason to explain a surge in support for them there, it’s clear this poll is completely off and it’s not just a “randomness” thing.
Anyways, the mid-term is next week. Be sure to read chapter 6 on Bayesian statistics by then…
Calgary Grit on
What’s the prior probability that we would read the chapter before the test?
made_you_look on
I’m sure this will help SC’s reputation. Considering the Green Party didn’t rush to throw this up on their site should be a pretty good indication as to the reliability of this poll.
WDM on