The federal election campaign has been, to paraphrase Jonathan Safran Foer, extremely long and incredibly close, and anyone who can make a confident prediction must be in possession of a finely tuned crystal ball. With so many polls, so many debates, and so many swings back and forth, how can we hope to sift out what really matters?
This sounds like a job for superforecasters. These uncommonly clued-in citizens—none of them political “experts” with public platforms—triumphed in the Good Judgment Project, a forecasting tournament that ran from September 2011 to June 2015. The project was funded by the U.S. government’s intelligence services, which identified a small group of people who were significantly better than paid pundits at predicting things such as foreign elections. The study was run by Philip Tetlock, a Canadian-born professor of psychology and political science at the University of Pennsylvania, whose new book Superforecasting, written with Canadian author Dan Gardner (Risk, Future Babble), is meant to arm readers with the same skills their superforecasters have honed.
In getting regular folks to assign odds to real-world events, Tetlock says, “We’re basically pushing people to do something that many initially think of as impossible”—to quantify unquantifiable events. Many superforecasters called the Scottish independence referendum and the U.K. election this year (although few foresaw an actual majority for David Cameron).
An extension of the tournament, now funded by the private sector, asks the question, “Which party will the next Canadian prime minister come from?” As of Oct. 7, entrants—including superforecasters and mere forecasters—collectively estimated that there’s a 48 per cent chance the PM will be a Conservative (i.e., Stephen Harper); the Liberals are at 37 per cent, and the NDP at 16 per cent (figures rounded up). According to Gardner, “For weeks, you’ve had this three-way tie, and, if you’re simply aggregating the polls, you end up throwing your hands up in the air and saying, ‘This is impossible to call.’ But superforecasters engage in a very rigorous, methodical, subjective judgment, so they can do better than that.”
We asked a few of the Canadian superforecasters for their tips in calling the election. Mark, a geophysicist in Ottawa (who asked that his last name be withheld), suggests casting a wide net when looking for data, so “you’re not dependent on one poll or source; disregard anything said by a strong partisan, regardless of the party.” The only exception to this last suggestion is Mark’s observation that NDP Leader Tom Mulcair and Liberal Leader Justin Trudeau appear not to have slagged one another off so mercilessly that they wouldn’t be able to shake hands and make up. Therefore, a coalition, from that point of view, remains a possibility. Debates, says Mark, are similarly useless: “You were hearing the same talking points. They weren’t actually true debates, and I think, in that case, the only time they have a really strong influence on the polls is when somebody really screws up or comes up with a short, punchy soundbite.”
As for the polls themselves, not all are created equal. For Brian Labatte, a Montreal-based superforecaster who works in marketing and business development for an engineering firm, sample sizes, margins of error, and polls’ provenance (“Would you trust Leger in Alberta, or do you feel they’re stronger in Quebec?”) are all important. The decay factor comes into play, too: Ask yourself whether the questions the poll has asked will still be relevant when the election rolls around. Labatte points to polls that focus on what values prospective voters believe their politicans share.
Superforecasters disagree about how much importance to assign Harper’s focus on the niqab. For Mark, what many have called a “dead cat” can be a game-changing “black swan, something that could solidify a certain percentage of the Quebec electorate and the Western redneck vote behind the same issue.” Labatte, meanwhile, says the media are too obsessed with this. He points to the strong presence of “Anyone but Harper” supporters in Quebec, and suspects that strategic voting is behind the swing in the polls there, as well as a growing uncertainty about the strength of the “protest-vote” NDP candidates who were elected last time out.
Superforecasters in tournaments often worked in teams, and Labatte suggests asking as many people as you know around Canada who they think will win the election—not whom they want to win—in order to get away from one’s own bias. Tetlock wants to take this branching out one step further: The next generation of his research, he says, “should focus also on supernetworkers, people who are really good at branching out their information search into disparate domains, linking to people who don’t know each other.”
For Patrice Odo, a Montreal actuary and superforecaster, the poll aggregator at ThreeHundredEight.com—which shows trends over time—is useful; he, Labatte, and Mark all foresee a close election with the Conservatives getting the most seats, but the Liberals forming a minority government. Odo cautions that the one sure thing in this election “is that it is completely unpredictable!” Then again, he would say that, Gardner notes. “One of the most basic aspects of the superforecasting mentality is intellectual humility.”