Measure the Person or Measure the Device

To measure the person or the device is arguably the critical choice for media measurement.

The choice made can shape what one understands about how people use media today and that choice ultimately affects our insights into how we think people will use media tomorrow.

Multi-platform-usage

The media measurement industry understands this, but can do better in explaining to everyone else how that choice matters.

Let’s take broadcast radio and streaming radio as an example.

In the biggest cities in the US, broadcast radio listening is measured by Nielsen’s Portable People Meter, a small passive pager-like device that a person wears on their body, much like…well a pager.

Imagine a person preparing a meal in the kitchen for a 30-minute period, listening to a radio station.  Mid-way through this 30-minute period, they have to step away from the kitchen, and away from the radio, to attend to a child playing outside for 10 minutes.  They return to the kitchen and continue listening to the radio. In this basic example, the meter collecting exposure to the encoded radio signal will indicate they listened to the radio for 10 minutes, stepped away for 10 minutes, and then listened again to the same station for a further 10 minutes – as the meter on the person’s body determines that the person was within earshot of the radio.

Now, using the same scenario, imagine the person is in the kitchen for 30-minutes, but this time is streaming Pandora through their smartphone.  In this scenario, the device (the smartphone), not the person, is being measured.  When the person steps away for 10 minutes to attend to their child playing outside, the device has no way to know whether someone is listening or not.  The end result will show 30 consecutive minutes of uninterrupted listening to Pandora on the person’s smartphone, regardless of whether there was one person or many persons listening to any of the streaming music.

Hence the biggest disadvantage to measuring the device.  It might be ‘playing’, but is anyone listening?  Is anyone watching?  Is anyone actively engaged with the content?

And yes, I recognize that the PPM is not foolproof either, but at least it measures exposure to an audio signal which is equated as listening.

Reviewing the streaming data will show an uninterrupted 30-minute period of listening.  It will likely be assumed that the person is paying full attention to Pandora’s music, even when other lifestyle data will indicate that people come and go from their audio playing device whether it is a broadcast radio or a streaming device.  Some will conclude that people spend longer periods of time engaged and listening to Pandora than their local radio station, which is also wrong.  Data points like this will then be used to support the continued push of some services over others because listening and time spent listening metrics that look similar, but aren’t, are compared to one another as if they are apples-to-apples.

Misuse of data is nothing new – politicians do this to their advantage, businesses do the same.  In my experience, it’s seldom as intentional as one might conclude, still it’s harder to correct a misstated fact than it is to get it correct in the first place.  But it happens.  It keeps researchers busy and it drives them crazy at the same time! And it’s a classic example of how media measurement data is used to support an argument without fully understanding how it was being collected.

Measurement in the web/mobile space is just as complicated and can be fraught with even more misconceptions.

In recent years, both Nielsen and ComScore have moved towards building more sophisticated user-panels to measure web usage in an attempt to measure across devices, which still remains as the holy grail for digital measurement.  But this approach is arguably open to misuse through measurement boosters from fake users.  Companies themselves can collect data from their own servers, as many newsrooms currently do, but even this is flawed as these systems struggle to differentiate a user that might access a site through four different devices and be counted as four different users.  Hence, panels of people become the preferred approach, despite the pros and cons that come with almost any data source.

Media measurement is a critical field to better understand media behavior and prompts us to think about a critical choice – measure the person or measure the device. Let’s also not forget the second part  – understanding how the data was collected and how it should be explained in the bigger media-using picture.

This is not about earbuds

If you show a ratings chart that pulls people out of their comfort zone, then inevitably their reaction is to focus on what is missing and what was not counted.  After all, that would explain the demise in the ratings.

From afar, this has been the reaction I have seen to the RRC‘s recent report on broadcast radio listening trends for US public radio, as well as trends for all radio.

The chart below is a wake-up call for the entire radio industry because it shows that there has been a steady (but also alarmingly quick) decline in the percentage of persons aged 6+ who listen to any measured media, which in this case is effectively any measured FM/AM/HD station.

Image

The chart shows that radio usage has fallen by 13% on weekdays in the past three years across the 30 markets measured by Nielsen Audio’s PPM.

This naturally alarms anyone working in radio.  As a presenter of this type of “bad” news for many years in many settings, the reaction from readers and from those in the room is part-defensive and part-wanting to understand.

The defensiveness or denial stems from the worry that one’s own livelihood is on the line, that the data confirms their worst fear that though their own media behavior hasn’t changed, the mainstream has, and that change is permanent.

This is a very human reaction, but as Mark Ramsey points out “In order to rally and fight a problem, we must begin by recognizing its existence“.  Holding onto the standard claims about the near universal reach of radio conveniently sidesteps the critical issue of engagement with radio.

The “wanting to understand” entails listening to each of the theories proposed that explain the ratings demise.  Some have credit, which should be acknowledged to aid understanding, but many can be debunked where necessary, in order to help people focused on what is real or not and what is within their control.

Don’t get me wrong, the ratings chart above effectively does not include headset listening; the workaround provided is simply not used by the ‘average Joe’.  But if you see someone wearing a headset, are you willing to say that person is absolutely listening to a local FM/AM station?  Probably not.

Yes, streaming is growing exponentially and it is the future for audio.  But take a 1-minute look at the Triton Digital Rankers report and you’ll see that the “Pandora train” has already left the station, leaving other pure-plays and broadcast networks holding only steady, and far, far behind, at best.

True, podcasts are not counted in the ratings.  Sure, it alters listening behavior so that I can listen to a show when I want to, but I haven’t seen any data yet that justifies the claim that if it weren’t for podcasts, my numbers would be much, much higher.

What’s left is what’s within your control.  Not how media consumption is measured.  Not about what content is or not counted. Not whether there was an election this time last year.  It’s about making what you do – audio, video, words – the very best it can be and the most compelling for your audience. Again and again.

Ratings tell us where we’ve been and can project where we’re going.  They can also show whether what we currently do has a future.  By all means, absorb them, understand them, and go ahead and poke holes in them, but don’t sit still in your comfortable seat and blame it on the earbuds.

Image