Where Is The Sabermetrics of TV Ratings?


I like TV By the Numbers. Many don’t. It’s a good source for ratings information and at least some ratings analysis. Now, the thing about the site is that it is completely amoral. It’s a horse-race site. It doesn’t try to find reasons why a network might be justified in renewing a good show — the way Fox found reasons, in real life, to renew Fringe despite its abysmal ratings. It just focuses on the number that matters most, viewers 18-49, ranks shows based on 18-49, and trains its readers on how to get some predictive value out of ratings: to understand that 18-49 ratings make the biggest difference to whether your favourite show will get canceled, or that a certain 18-49 rating might be acceptable on NBC but not on CBS. The site has helped to create what we might call faux-experts, commenters who will look at the 18-49 of Fringe or The Good Wife and announce that it’s doomed. But what it does is still pretty worthwhile in helping us understand what the ratings mean on a slightly more than superficial level.

Only slightly, though, and that’s where ratings analysis on the internet is still way behind analysis of, say, political polling. (For once, things that matter are getting more and deeper interest than things that don’t!) Nate Silver became famous in 2008 for taking the statistical-analysis techniques familiar from baseball, applying them to polls, and really getting the maximum amount of predictive significance out of polling. We don’t really have anything like that for TV yet. It wouldn’t be particularly difficult to do, for someone who knows how to read numbers: it would require looking at what shows have been canceled or renewed in the past decade or so, and figuring out what level of ratings predicts renewal or cancellation in different circumstances. And just as baseball and polling analysis takes into account all the outside factors that can skew the numbers, this kind of TV analysis would have to account for stuff like time slots, daylight savings time, even the amount of money a show stands to make from things other than advertising. (I don’t know that Warner brothers offered Fox a larger cut of the DVDs and other ancillary marketing of Fringe, but that kind of deal has been made in the past.)

And of course, it would require a more in-depth understanding of the meaning of demos. Right now most of the analysis seems to be stuck at “look at 18-49 and only 18-49.” Which is not a bad rule of thumb, because a) That is the demographic that matters most, and b) The decisions are made by network executives and advertisers, and they can be pretty superficial sometimes. (In other words, we may think “they couldn’t be so superficial as to think only viewers 18-49 have any value.” But maybe they are that superficial, in which case we have to think like a guy with a product to advertise.) But all this really tells you about is the “blowouts,” the shows that clearly will be renewed or clearly won’t be (unless the show with is Fringe). Anyone can look at a high 18-49 rating and know the show is safe, just as anyone can look at any single poll for a blowout election and know who’s going to win. But this doesn’t help with shows that are neither hits nor flops, which are the ones that are hardest to predict. When networks start taking all the factors into account, things that don’t usually “matter” may start to matter more than they normally do.

Sort of like Conan O’Brien’s poor performance with total viewers on The Tonight Show really did matter to NBC (according to The War For Late Night) once he was no longer completely crushing Letterman in 18-49. Does that mean total viewers really do sort of matter, or that they matter when developing a strategy to create broader appeal in every demographic, or that they matter to the affiliates with their older-skewing lead-ins and lead-outs? I don’t have the answers to any of these questions, and I’m not qualified to study the numbers and find out whether they provide answers. But until someone comes up with answers, the renewal of Fringe, or the renewal of some old-skewing show (Harry’s Law and The Good Wife could very well both be back next season) will not be predictable because the current models are really not predictive models at all.

Filed under:

Where Is The Sabermetrics of TV Ratings?

  1. Jaime,

    Good to know we've got readers up north, and the comparison to Nate Silver is flattering. Still, my guess is that our batting average in picking last year's renewals and cancellations was as good as his was predicting the congressional split in Nov 2010. Particularly if you judge our picks during the same time interval prior to the renewal or cancellation announcement as his before the election date.

    And while it does make intuitive sense that including lots more variables into the analysis would make things more accurate, in practice the relative adults 18-49 ratings explained more than 90% of the US broadcast scripted renewals and cancellations, far more than just the "blowouts", and most were clear far in advance of the announcements.

    BTW, the old skew of the Harry's Law audience is meaningless, it has the adults 18-49 ratings for renewal (on the weakling that is NBC), and I currently have it more likely to be renewed than cancelled.

    As for Fringe, my best guess, although Rupert Murdoch isn't returning my calls, is that Warner Brothers made Fox a crazy deal for another season, (ala, 'Sony & Til Death) to assemble the episodes for syndication. We clearly swung and missed on that, but I'm confident we won't have more than a handful of other misses this season.

    Keep reading!

  2. What matters also is whether the viewers have money – a show like The Good Wife skews older, but also skews wealthier, for example. Another essential point is whether the show brings prestige to the channel. Factor these in and renewal becomes easier to predict.

Sign in to comment.