To measure the person or the device is arguably the critical choice for media measurement.

The choice made can shape what one understands about how people use media today and that choice ultimately affects our insights into how we think people will use media tomorrow.

The media measurement industry understands this, but can do better in explaining to everyone else how that choice matters.

Let’s take broadcast radio and streaming radio as an example.

In the biggest cities in the US, broadcast radio listening is measured by Nielsen’s Portable People Meter, a small passive pager-like device that a person wears on their body, much like…well a pager.

Imagine a person preparing a meal in the kitchen for a 30-minute period, listening to a radio station.  Mid-way through this 30-minute period, they have to step away from the kitchen, and away from the radio, to attend to a child playing outside for 10 minutes.  They return to the kitchen and continue listening to the radio. In this basic example, the meter collecting exposure to the encoded radio signal will indicate they listened to the radio for 10 minutes, stepped away for 10 minutes, and then listened again to the same station for a further 10 minutes – as the meter on the person’s body determines that the person was within earshot of the radio.

Now, using the same scenario, imagine the person is in the kitchen for 30-minutes, but this time is streaming Pandora through their smartphone.  In this scenario, the device (the smartphone), not the person, is being measured.  When the person steps away for 10 minutes to attend to their child playing outside, the device has no way to know whether someone is listening or not.  The end result will show 30 consecutive minutes of uninterrupted listening to Pandora on the person’s smartphone, regardless of whether there was one person or many persons listening to any of the streaming music.

Hence the biggest disadvantage to measuring the device.  It might be ‘playing’, but is anyone listening?  Is anyone watching?  Is anyone actively engaged with the content?

And yes, I recognize that the PPM is not foolproof either, but at least it measures exposure to an audio signal which is equated as listening.

Reviewing the streaming data will show an uninterrupted 30-minute period of listening.  It will likely be assumed that the person is paying full attention to Pandora’s music, even when other lifestyle data will indicate that people come and go from their audio playing device whether it is a broadcast radio or a streaming device.  Some will conclude that people spend longer periods of time engaged and listening to Pandora than their local radio station, which is also wrong.  Data points like this will then be used to support the continued push of some services over others because listening and time spent listening metrics that look similar, but aren’t, are compared to one another as if they are apples-to-apples.

Misuse of data is nothing new – politicians do this to their advantage, businesses do the same.  In my experience, it’s seldom as intentional as one might conclude, still it’s harder to correct a misstated fact than it is to get it correct in the first place.  But it happens.  It keeps researchers busy and it drives them crazy at the same time! And it’s a classic example of how media measurement data is used to support an argument without fully understanding how it was being collected.

Measurement in the web/mobile space is just as complicated and can be fraught with even more misconceptions.

In recent years, both Nielsen and ComScore have moved towards building more sophisticated user-panels to measure web usage in an attempt to measure across devices, which still remains as the holy grail for digital measurement.  But this approach is arguably open to misuse through measurement boosters from fake users.  Companies themselves can collect data from their own servers, as many newsrooms currently do, but even this is flawed as these systems struggle to differentiate a user that might access a site through four different devices and be counted as four different users.  Hence, panels of people become the preferred approach, despite the pros and cons that come with almost any data source.

Media measurement is a critical field to better understand media behavior and prompts us to think about a critical choice – measure the person or measure the device. Let’s also not forget the second part  – understanding how the data was collected and how it should be explained in the bigger media-using picture.