The Ratings Game is a weeklong series exploring what the new world of TV ratings means for your favorite shows.
Almost every artistic endeavor in Hollywood comes attached to a ranking of some sort, a numerical report card that allows the town to declare said project a success or failure — and then gossip about it endlessly. Whether it’s Monday morning box office or the overnight Nielsens, everything gets judged. The one exception to this Tinseltown truism: Netflix. Three years after House of Cards announced its arrival as a programming powerhouse, the streaming-video pioneer has managed to mostly keep secret all but the most basic bits of information about how series perform with its subscribers. And it’s not just the media or the public in in the dark: Producers of Netflix originals claim to know precious little about who’s watching their shows. This almost unprecedented lack of data has left many in the entertainment industry frustrated, but also determined to finally uncover the numbers Netflix seems determined to keep hidden.
If there’s been a poster child for Hollywood’s annoyance at Netflix’s lack of transparency, it would be John Landgraf, the well-respected industry veteran who runs FX Networks. Since at least 2013, he’s been raising concerns about the fundamental unfairness of a major industry player avoiding any sort of independent assessment of its performance. Just before House of Cards launched in 2013, he predicted the company would never say how many people watched it because executives at the company “like the fact that they don’t have to have a report card,†implying the lack of data would allow them to avoid fallout from shows that don’t resonate with subscribers. Landgraf was right, of course. Shortly after the FX boss made his comments, Netflix content chief Ted Sarandos pretty much confirmed the then-fledgling programmer had no intention to jump into the ratings game. “There’s no business reason for us to … other than to create artificial pressure on a show to perform on a short timeframe,†Sarandos told Variety. “We don’t need to add that to the mix. There’s enough pressure in creating television as it is.â€
Netflix’s public silence on numbers has always had a certain logic to it. Ratings are primarily a tool for selling advertising time, or convincing cable operators to pay networks more in subscription fees. Netflix, which draws almost all of its income from that monthly fee it quietly deducts from your checking account each month, doesn’t have to worry about either. As one senior broadcast network executive reasons, “It is not at all in their interest to release significant ratings information. So why would they ever?†Netflix may have also learned a lesson from HBO. During its first two decades, the premium-cable trailblazer, whose business model is nearly identical to that of Netflix, would often brag about Emmys but rarely made a peep about ratings for its movies or original series such as Dream On and The Larry Sanders Show. But when The Sopranos started pulling in massive ratings in the early 2000s, network leadership at the time couldn’t resist bragging. Reporters who covered TV and other industry insiders soon began tracking the numbers for all of HBO’s shows, both good and bad. Today, despite the subsequent success of Game of Thrones and True Blood, some HBO insiders privately confess they wish they’d never opened up the Pandora’s box labeled Nielsen.
Perhaps more impressive has been Netflix’s discipline in avoiding the sharing of detailed information with the many big studios, stars, and producers it has partnered with to produce content. “As not transparent as they are to the press, that’s how not transparent they are to us,†one person involved in producing shows for Netflix told Vulture. “They would promise all the time to be more transparent. I heard for months, ‘We’ll give you the [ratings]’ … But they’d keep delaying. They wouldn’t show us.†Or, as another producer told us, “They’re not transparent to anybody.†Even Tina Fey claims not to know much about who’s watching Unbreakable Kimmy Schmidt. “We know that Ted [Sarandos] is pleased, which is great news, but we don’t have any actual numbers,†the producer/actor told journalists at a news conference for the show last winter.
While raw data for Netflix originals seems almost impossible to come by, the service does share some specific information with partners, according to more than a half-dozen industry sources Vulture contacted for this story. “I’ve never heard numbers expressed in millions or hundreds of thousands of views,†one studio executive says. “It’s more like early box office: ‘You’re off to this kind of start.’†A writer/producer who’s worked on a Netflix series said executives “would make allusions to how a show was performing†without getting too specific. “They’d tell you, ‘You’re doing better than this show on a linear network,’†he says. “Or they would offer demographic input. They’re willing to give you a sense of where your show is more popular than something else.†For the most part, however, “they dole out information on a need-to-know basis.â€
Netflix has slightly looser lips when it comes to shows it acquires — library content such as past seasons of New Girl or Parks and Recreation. But even then, it’s not by choice. “If a studio has enough leverage to negotiate it with Netflix, they’ll get a data readout from them,†says one network insider. “It’s monthly, and it’s basic stuff, like how many people start every episode.†One person who’s seen such a report says it was only moderately helpful: “It was just streams to date, and it was only for the shows we produce.†Most frustratingly, our network insider notes, “it doesn’t tell you much [about] how your shows do opposite others.†Translation: 20th Century Fox TV, which produces and licenses Family Guy to Netflix, might conceivably know that, say, 1 million people streamed the show in October. It has no idea, however, how that compares to Warner Bros. TV’s Friends or NBC Universal’s The Office.
Given how much Hollywood types love to gossip, surely executives are busy trading this (limited) information with each other, right? Yes and no. “There’s a lot of sharing of notes,†one person who works for a major Hollywood conglomerate says. But because the data is secondhand, it’s not all that useful: Execs don’t know for sure if their rivals are telling the truth about information, or if the kind of data Netflix is supplying one content partner is identical to what they’re getting. What’s more, Netflix is very strict about making all of its partners sign non-disclosure agreements. “So any data you get from Netflix, you’re not allowed to do anything with it,†one industry insider says. Our former Netflix writer says even talking to others contracted to the company wasn’t much use: “Every once in a while I’d bump into another producer of a Netflix show. But they wouldn’t know much more than I did.†Perhaps the biggest indicator of how tight a grip Netflix maintains on its data: Even the biggest yentas in Hollywood — agents — insist they’re pretty much clueless about who’s watching their client’s shows, either on Netflix or on streaming rivals Amazon and Hulu. A partner at a leading talent agency — a decades-long veteran who usually knows where all the bodies in Hollywood are buried — was blunt: “I don’t have real data on any of those three services,†he said.
This is not to say Hollywood has simply accepted Netflix’s no-numbers policy as law. “Everybody’s trying to come up with solutions, though right now it’s very limited,†one network-research person told us. Some third-party companies are already experimenting with using the sort of surveys Nielsen uses to measure traditional TV shows to gather data about shows on Netflix (as well as Amazon and Hulu). Luth Research, based in San Diego, is regularly monitoring thousands of “Netflix Families†to track what they’re watching — but, as of this spring, they can only monitor viewing that takes place via computers, tablets, and phones. Nielsen a year ago announced plans to track Netflix viewing on TV, but insiders say such plans have been slowed by technical issues. (It hasn’t helped that Netflix appears to be purposely changing audio coding on its shows in a bid to thwart outside companies from measuring its content.) Network sources says they’re confident workarounds will be discovered, and that within a few years more reliable data will start streaming out.
In our conversations with sources, several insiders also relayed secondhand reports of studios taking a page out of Netflix’s playbook by devising algorithms of their own, formulas designed to estimate how various series are performing within the Netflix universe. This is possible because of the information the company does share with program providers, namely those monthly library reports detailing how acquired shows are doing. As streaming gets more mature, studios are gathering more and more data; they can compare how a classic series such as M*A*S*H does when it first arrives on Netflix to its performance a year later. Studios can also look at the streaming they get from other outlets. While Amazon Prime guards its data just as jealously as Netflix, networks and studios get a relative wealth of information from Hulu. That’s because showbiz conglomerates Comcast, Disney, and 21st Century Fox are all partial owners of the service, and because Hulu sells advertising time on its streams. None of this makes for a perfect or wholly scientific solution, but it’s giving the industry a better idea of how viewing works in a long-tail ecosystem.
Another way the town guesses how its shows are doing on Netflix is by judging the company’s own behavior as a content buyer. “The best way to gauge is by looking at how they price shows,†our agency source says. “How much do they want something?†The agent points out that Netflix is currently “bidding hard†to renew its output deal with the CW, which expires this spring. That, he says, is a clear signal CW shows perform well on Netflix and have value. Other industry insiders agree with the notion that Netflix tips its hand by how it negotiates for content, particularly when rights to individual series come into the marketplace: Sources say it’s not unheard of for studios to basically test the worth of their shows by letting Netflix (as well as rivals Hulu and Amazon) know that streaming rights to a series may soon be available for purchase — and then seeing how much each is willing to pay.
Figuring out who’s watching what on Netflix isn’t just a matter of curiosity or bragging rights for Hollywood. Actual money is at stake, too. While ratings are mostly used to set ad rates, they’re also often key in determining the monetary value of a show — and the people who make it. Our former Netflix producers notes that without knowing how well a show does over a two- or three-year timeframe, or even longer, “it becomes completely impossible to do any sort of oversight when it comes to royalties or residuals.†It also makes it very difficult for actors and writers to know whether they’re being fairly compensated for their work should their Netflix series last several seasons. Stars on traditional network shows have ratings data to hold over the heads of their studio and network employers: Back in 2002, the cast of Friends, for example, were able to get a desperate NBC and Warner Bros. to pay them a whopping $1 million per episode to keep the show on the air one more season. Their agents felt safe threatening to walk away from the show because the Nielsen report card made it clear NBC couldn’t afford to lose the show. Several industry insiders we spoke to told us Netflix holds on to its data so tightly in part to avoid a similar scenario, both with actors and with studios licensing past seasons of shows. “Not knowing gives Netflix all the leverage,†one studio numbers-cruncher told us.
Despite Hollywood’s annoyance at not knowing who’s watching what on Netflix — and the very real financial implications — our conversations with industry insiders revealed a surprising acceptance of the situation, or at least no real ire. That may be because some believe Netflix and other streaming services won’t be able to live in their metric-free bubble for long. After all, we’re just a few years into the streaming era of TV, making it an incredibly young platform. “They’re operating under much more beneficial conditions right now,†one top-level TV executive says. “There’s no ratings pressure, and there’s the benefit of being the new kid. But ultimately, there’s going to come a turning point. They won’t be the new, shiny object that everyone is obsessed with. And then they’ll face the scrutiny all of us face.†That’s probably true, and no doubt many of Netflix’s linear-TV rivals will be happy when that day arrives. For now, however, some in the industry are actually enjoying what may be a Camelot-like moment at Netflix, where nothing gets canceled and every show is a hit. “I feel pretty good not knowing about shitty ratings,†one agent laughs. “I’m not dying to hear more bad [news] every morning.â€