The Future of the Ratings Panel
Jennifer Hessler / Bucknell University
A defining feature of the Nielsen ratings has always been that they derive from a statistically sampled panel of real viewers. In his 1966 promotional speech titled “If Not the People…Who?” A.C. Nielsen Jr. described audience measurement as akin to a democratic election, the ratings constituting “the voice of the people” and a “mirror of public taste.”[1] Eileen Meehan argues that in its early days Nielsen strategically employed this rhetoric to characterize the ratings as quintessentially American and good for free market economic prosperity, while also absolving itself of any responsibility for the ratings’ negative affect on program quality.[2] Even though media scholars have debunked the idea that television ratings are an accurate reflection of public taste,[3] the ratings’ reliance on a viewer panel has still shaped their epistemological value: one the one hand, making them (somewhat problematically) dependent on panelists’ cooperation, and on the other hand, accruing them the credential of direct “audience intelligence.”
As the age of big data reimagines audience analytics, the ratings panel is at the center of industry debates about how to define audience intelligence. Nielsen itself is turning toward new forms of machine learning and artificial intelligence (AI) to create and interpret viewing data. Essentially machine learning and AI endeavor to “objectify” audience data while also supposedly “individualizing” it at large scales. For proponents, these mechanisms beg the question: if machines can supposedly replicate human decision-making processes, is the direct input of real viewers necessary for the production of “audience intelligence”? But on the other hand, Nielsen also argues that their ratings being derived from a panel of real viewers has advantages over their competitors’ data sets. In the disarray that currently characterizes the digital data market, Nielsen says that their viewer panel gives them the ability to bind and re-embody data, resulting in a more accurate capture of real media engagement.
The ratings panel has been at the center of the methodological rivalry between Nielsen and one of its main competitors, comScore. ComScore was founded in 1999 by Gian Fulgoni and Magid Abraham, formerly of Information Resources Inc. (IRI), an integrated big data and predictive analytics firm. In 2002 comScore partnered with Media Metrix, which utilized a PC meter to measure internet traffic, somewhat similar to the method used by Net Ratings (acquired by Nielsen in 2007). And in September 2015, comScore acquired Rentrak, a company that specialized in box office data and in aggregating television audience data from set-top boxes and Digital Video Recorders. With the merger, comScore became an industry leader in digital audience metrics.
Being born of the digital age, everything from comScore’s tracking technologies to their panel recruitment practices are created for an online environment. ComScore places beacons and trackable tags throughout the online content they monitor. When internet users/viewers visit the tagged websites, the beacons store a tracking cookie in the user’s computer memory. Meanwhile, comScore also has a panel of around two million people, who they recruit through a combination of randomized digital dialing and volunteer surveys. Panelists agree to run comScore’s background monitoring software package on their devices, which tracks everything they do online. ComScore then compares the data it collects on total web traffic to the tracking data from its panel to decipher more specific demographic information. In an interview with Digiday, former CEO Bryan Wiener stated, “We believe in panels. The biggest difference between us and [Nielsen] is we believe that we’re data-first and the panel is used to inform the data set versus the panel being at the crux and using the data at the outskirts.” Rather than being the source of its data (like it is for Nielsen), comScore’s panel serves a secondary referential function.
While often thought of as Nielsen’s more digitally competent brother, comScore has its own shortcomings. Privacy advocacy groups criticize the firm’s practice of storing tracking cookies in the computers of users who have not agreed to be a part of their panel, many of which are likely incognizant of their involvement. ComScore’s panel is also criticized for not being demographically representative, and the obscurity around its sampling means it’s difficult to know how fairly the firm is measuring underrepresented populations. And lastly, comScore’s metrics emphasize viewing that occurs through the internet or connected devices; the only audience data they collect for broadcasting comes from set-top-boxes, which makes it unrepresentative of the linear audience as a whole.
On the other hand, Nielsen’s ratings, remnant of the broadcast era, use the live broadcast as the base of their viewing metric before adding on digital viewing, and they rely more centrally on a panel of real viewers. Nielsen argues that this “tried and true” method allows them to more productively ground the disarray of digital data, to connect the gaps that code and algorithms leave in understanding viewers’ engagement with media. In an interview, former Nielsen Senior VP Jessica Hogue explained, “As Nielsen continues to move into cross-platform measurement, working with a greater abundance of data, the panel will become increasingly important. Large sets of household data make the panel an invaluable tool for personification.”[4] The term “personification” alludes to a unique epistemological value in the subjectivity of the viewer panel. Nielsen’s former Executive VP Megan Clarken elaborates on this when she says that set-top boxes and web-trackers are, to a certain degree, dumb devices that represent the machine’s footprint, not necessarily the viewer’s engagement. She says, “Until a [set-top box or smart TV] can identify itself and give you a data set that identifies its relationship to everybody in the home,” it will not provide the same value as a ratings panel.[5]
Nielsen continually emphasizes the advantages of panel-based measurement over big-data sets for understanding demographic differences. In a study focusing on Fox’s Empire (2015-), Nielsen found that while Empire ranked 16th in viewership using data from Nielsen’s nationally representative panel, in which “there’s a focus on race and ethnicity as well as making sure that we’re representing across the geography,” when using return path data or the big data sets from the same period (December 2018), Empire moved down to 38th in viewership. Nielsen’s Senior VP, Kelly Abcarian further explains that 75% of Empire’s audience is multicultural, and those audiences drive the show’s ratings. She argues, “If you’re not ensuring that the representation is there, it can drastically change the on-screen talent, it can change the programming lineup, and it can change the way advertisers think about that content.”[6] While one must be weary of the bias of Nielsen’s own study, it nonetheless demonstrates the stakes of how the industry conceptualizes audience intelligence for the future of diverse representation.[7]
At the Coalition for Innovative Media Management’s 5th Annual Cross-platform Media Measurement & Data Summit, the final day’s discussion ended on the topic of the ratings panel, with comScore Chief Product Officer, Manish Bhatia, and Nielsen’s Abcarian each summarizing their firm’s stance. ComScore’s Bhatia argued, “As good as a panel is, there is a limit to how finely you can slice that panel… There’s only so much juice you can take out of a lemon.” And Abcarian countered: “While comScore’s tunnel vision is scale, Nielsen’s focus is also on quality.”[8]
The vastly different logics underpinning Nielsen’s and comScore’s methods is perhaps why so many media companies have found value in subscribing to both firms during this period of digital transition. Currently, the companies are each working to address their metrics’ limitations. As reporter Tim Peterson characterizes it, “Both companies are racing to establish strengths in each other’s domain. ComScore is pressed to provide the in-depth person-based measurement that Nielsen’s panels provide, while Nielsen must aggregate more data to augment its panels and provide more minute measurement at the device level.”[9] As the two companies work from different directions, toward cross-platform measurement, it is likely that they will collide in the middle, and it will be left to the industry to decide which audience metric suits their needs. But beyond industry utility, how the digital television market comes to conceptualize the scientific notion of audience intelligence will have lasting implications for the future of audience surveillance, demographic representation, and television programming.
Image Credits:
- At stake in Nielsen and comScore’s rivalry is the very future of the ratings panel.
- A central part of evolving digital markets, audience measurement has been reconceptualized through the capabilities of big data.
- ComScore Headquarters
- Nielsen’s ratings panel, a remnant of broadcast history, is still the foundation of Nielsen’s metrics.
- Arthur C. Nielsen Jr., “If Not the People…Who?” An Address to the Oklahoma City Advertising Club. A.C Nielsen Company. Chicago, 1966. (Edgar Kobak papers, Library of Congress) [↩]
- Eileen Meehan analyzes this speech to demonstrate how Nielsen drew on Cold War rhetoric to characterize the ratings as quintessentially American and good for economic prosperity; the implication being that to critique the ratings methodology or their results was unpatriotic. Why TV Is Not Our Fault: Television Programming, Viewers, and Who’s Really in Control (Oxford, UK: Rowman & Littlefield, 2005), 16. [↩]
- Scholars like Eileen Meehan, Robert McChesney, Ien Ang, and Todd Gitlin have debunked this idea that TV ratings are a mirrored reflection of public taste by pointing out that, on the one hand, audiences are choosing among a relative narrow selection of commercially motivated choices in the first place, and on the other hand, sampling practices tend to over-represent some viewing groups and underrepresent others. Eileen Meehan, Why TV Is Not Our Fault: Television Programming, Viewers, and Who’s Really in Control (Oxford, UK: Rowman & Littlefield, 2005); Robert W. McChesney, “The Market Uber Alles,” The Problem of the Media: U.S. Communication Politics in the 21st Century (New York: Monthly Review Press, 2004), 175-209; Ien Ang, Desperately Seeking the Audience (London: Routledge, 1991); Todd Gitlin, “By the Numbers,” Inside Prime Time (London: University of California Press,1983), 41-48. [↩]
- Jessica Hogue, Nielsen Senior Vice President of Digital Client Sales and Services, Personal interview with the author, 27 April 2018. [↩]
- Jon Lafayette, “7 Things You Need to Know About Nielsen’s New Tool,” Broadcasting & Cable, 16 March 2018. [↩]
- Discussed (at 29:00) in “The Database: How Addressable Advertising is Personalizing the TV Experience,” Nielsen podcast episode 31, October 8, 2019, https://www.nielsen.com/us/en/insights/podcast/2019/the-database-how-addressable-advertising-is-personalizing-the-tv-experience/?utm_source=twitter&utm_medium=organicsocial&utm_content=nielsen&utm_campaign=Global+Media. [↩]
- Michael LaSardo, Vice President of Katz Television’s Station Solutions, offers an in-depth explanation of how Nielsen and comScore account for viewer demographics in local markets. Nielsen’s ability to supplement census-level data with demographic information provided by their viewing panels enables them to account for the demographics of individuals, opposed to comScore’s practice of placing whole households into single demographics groups. “Spot the Differences: Nielsen and Comscore,” Katz Media Group, 6 January 2020, https://blog.katzmedia.com/on-measurement/nielsen-comscore-spot-the-differences-series1. [↩]
- “Summary of 2.7.19 CIMM Cross-Platform Video Measurement & Data Summit,” Coalition for Innovative Media Measurement, 9 April 2016. [↩]
- Tim Peterson, “Comscore and Nielsen are Racing to become the One True Cross-platform Measurement Provider,” Digiday 2 January 2019, digiday.com/marketing/comscore-nielsen-racing-become-one-true-cross-platform-measurement-provider/. [↩]