Phil Birnbaum, who along with “Tom Tango” is probably one of Canada’s two great gifts to quantitative analysis in sports, has been studying the NHL over the past few weeks. It was only after a second or third reading of his series breaking down luck versus skill in the NHL standings that I was able to really grasp what he was saying. I’m a fluent speaker of basic stats-ese, but not a native. Phil is a pretty approachable explainer of things (including some of the things devised by Tango), so usually I don’t have to bash myself over the head too hard with his findings. But I didn’t see how interesting the message was until now.
Probably all hockey fans know instinctively that the introduction of the shootout has injected a fair amount of randomness into the year-end NHL standings. Birnbaum, looking at the shootout-era data, has now shown just how much. In the old NHL that still had ties, it took an average of 36 NHL games for a team’s actual talent to become as important to its standings position as sheer randomness. “Talent” is defined here as repeatable ability, ability relevant to prediction: after 36 games, your team’s distance in the standings from .500 would be about half luck and half “talent”, and that would be reflected in your guess as to how they would do in the next 36 games (assuming nothing else about the team had changed). Over a full season, we could be confident that there was little randomness left in the ordering of the teams in the league table.
But in the new post-ties NHL, Birnbaum notes, the standard deviation of standings points has shrunk from about .2 per game to .15. That change doesn’t look terribly dramatic, but… how to put this in something close to English? For a whole season, the amount of variance to be divided between talent and randomness depends on the square of the standard deviation. The amount of variance attributable to randomness is about the same as before, so the implicit effect of talent on a team’s place in the standings is much lower—so much lower, in fact, that it now takes 73 games to gather as much information as 36 games gave you before. Over a full NHL season, these days, luck ends up being still almost as important as ability.
And over a 48-game season… you can see where this is going. The final league table this year will be more a product of randomness than of talent. Birnbaum has done some simulations of the lockout-shortened season to illustrate how wacky ‘n’ weird we might expect the standings order to look when the dust settles, and you should look at those results if all of this sounds interesting to you. But the plain-English lessons, as I see it, cut in two directions.
We should be aware that predictions about standings for this shortened season are not much good. The very best teams will be ranked high, and the very worst low: but for teams anywhere near the cutoff for the playoffs, the chances of making it into the postseason will be a lot closer to 16 out of 30, a pure crapshoot, than our lifelong experience of hockey might lead us to think. Anyone who has made forecasts concerning the season will have a few clangers attributable to the large quantum of randomness in a short season with shootouts.
And after the season is over, we still need to remember how information-poor the 2012-13 league table will be. There will be teams falsely perceived as newly overpowering because they attained a high playoff seeding, and teams falsely perceived as being in crisis because they were at the bottom of their division.
There’s a deeper question here: what does Birnbaum’s reckoning indicate about the health of pro hockey overall? The “loser point” handed out for victories in overtime and in the shootout is still very controversial. The effect, even over a regular 82-game season, is to dilute—some would say “pollute”—the true ability of teams with a heavy dose of randomness or statistical noise. And we have certainly seen examples of clubs making the postseason by virtue of efficient collection of “loser points”.
On the other hand, this creates parity, practically by definition. On the whole, as the lowered standard deviation of standings points suggests, more teams are close to the middle of the table than before. And this arguably makes each individual regular-season game more meaningful and exciting. The point of the shootout is to put a standings point at stake in one supreme moment, and from that standpoint it works well. Shootouts are thrilling, especially when seen in person.
The presence of randomness in sport is an evil only in excess. The math Birnbaum uses to separate ability from luck was devised by Tango, and it works just as well for other sports, allowing one to differentiate, quantitatively, between ones that are high in randomness and ones that are low. The canonical low-randomness sports, like basketball and tennis, have highly predictable game outcomes. There cannot be any question that this is a problem for them: it’s a problem that there are few upsets in tennis, and it’s certainly a problem that the quality of NBA teams is apparent quite early in the schedule, making much of the season a chore.
High-randomness sports, on the other hand, will make it harder for dominant teams or individual competitors to appear. I haven’t seen or done the math, but golf must be a high-randomness sport; barring those periods in which God decides to take a name like “Tiger Woods” and revisit Earth for a while, there are dozens of players, perhaps a hundred or more, who might win any particular major. One can easily spend four days watching a PGA Championship only to see Ron Fartburg from Piehole, Oklahoma win the thing. Without any possible doubt, one reason the Masters Tournament is so successful and distinctive is that it is a low-randomness event in a high-randomness sport. It filters out the Ron Fartburgs by imposing crushingly high entry criteria (that’s why they’re the Masters!), and it is played on the same course every year, always rewarding the same bundle of particular skills and the same course-specific knowledge.
My personal view is that the “loser point” was an appropriate thing to add to the NHL regular season. The implementation is not ideal, because the league hands out three points in total for a game that goes into overtime, and that can create a situational incentive for teams to play for a regulation tie in the third period of a close game. There is strong statistical evidence that this does happen. So the argument that the league should give three points to a regulation-time winner and zero to a regulation-time loser is overwhelmingly strong, and in fact I do expect such a system to come about eventually.
But the short regular-season overtime and the shootout themselves, which make the NHL regular season less like the prolonged march toward foreseeable outcomes that is the NBA season, are probably good for the game—especially given that in the playoffs, NHL hockey switches to a different mode, one with potentially infinite sudden-death 5-on-5 overtime. This is a lower-randomness mode, a mode in which the superior team is more reliably rewarded. That seems appropriate for a playoff tournament, and it helps preserves some of the theoretical potential for the creation of consistently elite teams and the formation of dynasties.
We really can have it all, you see: there is no principle that says playoff hockey has to be exactly the same as regular-season hockey. Of course, it never was exactly the same, because unbounded sudden-death overtime has never been considered practical for regular-season play. The arch-conservatives who bewail the demise of the hard-fought regular-season tie are presumably the same people who relish the nerve-rending drama of a double-overtime playoff game. I like that the NHL has blundered into a “best of both worlds” situation; I like that there are two slightly different kinds of hockey played on different pages of the calendar, with a third set of circumstances for international events. It should be natural for Canadians, who will play hockey wherever there is any flat surface larger than a square yard, to believe that there is nothing wrong with letting the essence of the game be expressed in diverse ways.