Where Is The Sabermetrics of TV Ratings?

I like TV By the Numbers. Many don’t. It’s a good source for ratings information and at least some ratings analysis. Now, the thing about the site is that it is completely amoral. It’s a horse-race site. It doesn’t try to find reasons why a network might be justified in renewing a good show — the way Fox found reasons, in real life, to renew Fringe despite its abysmal ratings. It just focuses on the number that matters most, viewers 18-49, ranks shows based on 18-49, and trains its readers on how to get some predictive value out of ratings: to understand that 18-49 ratings make the biggest difference to whether your favourite show will get canceled, or that a certain 18-49 rating might be acceptable on NBC but not on CBS. The site has helped to create what we might call faux-experts, commenters who will look at the 18-49 of Fringe or The Good Wife and announce that it’s doomed. But what it does is still pretty worthwhile in helping us understand what the ratings mean on a slightly more than superficial level.

Only slightly, though, and that’s where ratings analysis on the internet is still way behind analysis of, say, political polling. (For once, things that matter are getting more and deeper interest than things that don’t!) Nate Silver became famous in 2008 for taking the statistical-analysis techniques familiar from baseball, applying them to polls, and really getting the maximum amount of predictive significance out of polling. We don’t really have anything like that for TV yet. It wouldn’t be particularly difficult to do, for someone who knows how to read numbers: it would require looking at what shows have been canceled or renewed in the past decade or so, and figuring out what level of ratings predicts renewal or cancellation in different circumstances. And just as baseball and polling analysis takes into account all the outside factors that can skew the numbers, this kind of TV analysis would have to account for stuff like time slots, daylight savings time, even the amount of money a show stands to make from things other than advertising. (I don’t know that Warner brothers offered Fox a larger cut of the DVDs and other ancillary marketing of Fringe, but that kind of deal has been made in the past.)

And of course, it would require a more in-depth understanding of the meaning of demos. Right now most of the analysis seems to be stuck at “look at 18-49 and only 18-49.” Which is not a bad rule of thumb, because a) That is the demographic that matters most, and b) The decisions are made by network executives and advertisers, and they can be pretty superficial sometimes. (In other words, we may think “they couldn’t be so superficial as to think only viewers 18-49 have any value.” But maybe they are that superficial, in which case we have to think like a guy with a product to advertise.) But all this really tells you about is the “blowouts,” the shows that clearly will be renewed or clearly won’t be (unless the show with is Fringe). Anyone can look at a high 18-49 rating and know the show is safe, just as anyone can look at any single poll for a blowout election and know who’s going to win. But this doesn’t help with shows that are neither hits nor flops, which are the ones that are hardest to predict. When networks start taking all the factors into account, things that don’t usually “matter” may start to matter more than they normally do.

Sort of like Conan O’Brien’s poor performance with total viewers on The Tonight Show really did matter to NBC (according to The War For Late Night) once he was no longer completely crushing Letterman in 18-49. Does that mean total viewers really do sort of matter, or that they matter when developing a strategy to create broader appeal in every demographic, or that they matter to the affiliates with their older-skewing lead-ins and lead-outs? I don’t have the answers to any of these questions, and I’m not qualified to study the numbers and find out whether they provide answers. But until someone comes up with answers, the renewal of Fringe, or the renewal of some old-skewing show (Harry’s Law and The Good Wife could very well both be back next season) will not be predictable because the current models are really not predictive models at all.