free
My friend Avinash Kaushik posted a wonderful article the other day about the importance of analysts to have a skeptical nature, and I absolutely agree with him. Skepticism, along with fact-checking, and a strong urge to take a step back to look at things from the larger perspective, is the key trait of anyone working with media strategies.
But I want to expand on his article because there are several types of graphs that I see all the time, each painting a completely misleading picture. And each one of these is dominating the media landscape, and is constantly used in presentations at pretty much all of the big media conferences.
So let's talk about this.
The first truly misleading graph is the one most people use to indicate market share of either their own business, their audiences, or the things that we use to get our publications to the market.
For instance, take a look at this graph:
Here you see two different things/products/whatever and how they changed from 2011 to 2016. So, what conclusions do you get from this?
The conclusion most people come to is that the yellow market has experienced catastrophic decline, while the red market is dominating more and more.
Right? Eh... no. What if I told you that the yellow market hadn't declined at all? And to prove this, here is the exact same graph but using the raw numbers instead of a percentage.
What's actually happening here is that the market overall has expanded. The yellow market has experience only a minor growth, while the new red market has created an entirely new market on top of the old one.
You see how bad this is? The first graph forced you to come to entirely wrong conclusions.
So, one of many places where this is happening is when people talk about the rise of mobile. For instance, people keep talking about how laptop computers are dead, and the graph they use is the one below:
What you see here is the same as before. As a percentage, mobile has been growing rapidly over the past five years at what looks like the expense of laptops. And if you see this graph (or those similar that are widely circulated by media executives) you may indeed think that laptops are dead.
What makes this graph even worse is when it is backed up by graphs showing volume of sales, where, again, mobile is dominating. This should not come as a surprise to anyone, since we buy a new mobile at a much higher frequency than laptops... especially today where we have so many devices to play with.
So, are laptops dying? Is it game over for desktop computing? Nope... not even close.
Because here are the exact same numbers, but this time drawn using their real data instead of as a percentage. And what you see here is that laptop use per person is the same today as it was five years ago.
The growth of personal laptop usage may have peaked, at slightly less than 3 hours per day, but it shows no sign of decline. What has happened instead is that we now have a new mobile market on top of the old one. It's not killing the laptops. It's extending it.
Think about how many times people have told you that mobile is killing laptops over the past few years, both at media conferences and on Twitter. It's just insane how many have been fooled by looking at percentages instead of the actual numbers.
Don't be one of those people. Always insist on seeing the real numbers.
There is, however, an even worse example than this. And that's when we see studies from newspaper associations. Almost all of them are using completely misleading graphs by default. Either because they don't know any better or, worse, because they are trying to hide the decline that we all know is happening.
The main reason why these graphs are so bad is because not only are they based on total percentages, they are also leaving out critical data.
Think about it like this. Imagine your market was defined by three types of audiences. Print, digital and not reached. Not reached are the people who have stopped reading newspapers as we know them.
Now imagine that we map this change over the past five years, we might see something like this:
What we see here is that the market overall is going up (due to the growth in population), but the share of people who don't subscribe to newspapers is increasing. For newspapers, their market is in heavy decline, and even though digital is growing, it in no way makes up for the decline.
Sound familiar?
But then the newspaper associations do a study, and instead of looking at the market as a whole, they decide to only look at their remaining market. In other words, they decide to simply ignore all the people who have been lost, and then map the rest as a percentage.
The result is a graph that looks like this:
Now we have the same problem as before. This graph is not only completely useless, but also completely misleading. Now, you no longer see any sign that the market is in trouble, and while digital is growing, it looks like print is still going strong.
This is terrible.
One example of this was when the Canadian newspaper measurement agency Vividata published this study:
Just look at this. This graph actually makes it look like newspapers are winning. They are up from 77% reach to 81% reach. And even the magazines are doing fantastically. Sure they are down by a bit, but it's nowhere close to anything that could threaten their future.
This is great. The Canadian newspaper industry has apparently found a way to keep winning with print. No problem here. Right?
Eh... no. The reality is, of course, that the media industry in Canada is in just as much trouble as newspapers anywhere else in the western world, and that their circulation and advertising revenue is in a terrible state... like what we see here:
I'm not saying that Vividata's data is wrong. It's probably an accurate reflection of what they measured. But what I am saying is that they designed the study to only look at things that had no real use for the challenges newspapers are faced with.
As they say:
Seventy percent of newspaper readers still read a printed edition daily. That's down from 90% five years ago. While print remains the leading source for most newspaper readers in Canada today, digital and cross-platform continues to grow.
No. Just no. This is an idiotic way to look at the data.
The future of the newspaper industry is challenged by external factors, so it makes absolutely no sense to do a study that only looks at the internal factors. This is stupidity at its worst, and it has a serious consequence.
When the Spiegel's Innovation report was leaked last week, one of the key problems they found was that they 'lacked a sense of urgency'. And of course they did. If you are constantly being told print is still leading and digital is growing, you don't feel any need to change. Things sound like they are going fine.
But this is not the reality.
So, be skeptical about newspaper studies. Almost every one of them is disastrously misleading... and especially so if they are based on percentages. Whenever I see a newspaper study that has percentages in it, my red warning lights begin flashing.
Another huge problem that we see with studies about the media industry is that we often see that studies that are measuring one thing are used to prove something completely different.
One example is this slide from BBC's Esra Do?ramaci at the News:Rewired conference.
I have lost count of how many times I have seen media people use this study in relation to video, and people were absolutely lapping it up at the conference. Several people tweeted that video was the only thing to focus on in the future.
But this is not what that study is saying. The Cisco study has nothing to do with video consumption. It doesn't tell us anything about whether people actually watch video more than before in relation to the media.
There are a number of reasons for this.
Firstly, the Cisco study is looking at the volume of network traffic going through their routers for all internet traffic as a whole. It's not looking at video consumption specifically.
This alone is highly misleading. You think that an increase in video use means people also watch more video, but that's not necessarily true.
Consider this.
Imagine that it's 2010, back with poor wifi and relatively slow mobile devices and you look at the consumption patterns of a single person. So, you have a person reading 10 articles, at 2 MB, and spending 2 minutes reading each. Then this person also watches one video, at 360p (again because of slow connections), at 16 MB, spending 6 minutes.
What you get is this:
As you can see, 44% of the data usage and 23% of the time spent goes to video, the rest goes to the articles.
Now fast forward to 2016. We now have 4K and amazingly fast wifi connections, so now we have got this:
You see what's happened here?
In this example, the consumption pattern stayed the same. This person is still only spending 23% of his time watching video, but look at the data usage. It's now 96% going to video.
And this is what the Cisco study is predicting. It is looking at the size of the video files and it's projecting how much data that will require in 2019. Cisco is not talking about time spent.
Mind you, I'm not saying that video isn't growing. We all know that it is. But it's not 90% of the internet. The idea that many executives have that video in the future will be 90% of people's time spent is completely misleading.
The problem we have today is that we have no good studies about how much time people actually spend watching video on video sites. All the studies we have seen are either based on video views (which has nothing to do with real consumption time), or based on video volume, which includes all the time we spend watching on-demand TV on Netflix etc.
Neither really tells us anything about how people consume the content from newspapers or magazines.
We also see 'attention' studies, but I have yet to see one that really measures this correctly. What you want to measure is when people are really reading an article or really viewing a video. And you want to measure the completion rates for both. The studies I have seen either don't do this correctly, or only measure one but not the other... leaving us with nothing to compare with.
So, again, be very skeptical about video statistics. What we have today is very likely to be misleading in a big way.
And of course, this isn't just about video. We see so many examples where studies are looking at one thing, but people think it applies to something else. And, again, the media industry is very bad at this.
Let me give you another example.
Imagine that we wanted to do a study and I asked you: "Do you read newspapers?" What would you answer?
You see, if you were from the older generation, this question could only mean one thing. It meant you were reading a printed collection of stories, within a certain type of focus, in a certain format, delivered on a daily basis.
Or in other words, it meant you were reading something that looks like this:
This was the reality of the old world of media. Everything was sharply defined and separate from each other. A newspaper was very different from a magazine, which was very different from radio, which again was very different from TV.
Asking people if they were reading a newspaper always resulted in a clear and unequivocal answer.
But now look at the media as we know it today. The first problem we have is that our many formats of media are now part of the same mix. When you go to the New York Times, you are just as likely to watch a video as you are to read an article. So, are you reading a newspaper?... or are you watching TV?
Then look at the consumption patterns. In the past, reading a newspaper actually meant sitting down with this package of news and reading or looking over a substantial part of the articles. We don't do this anymore. Today we read articles via links, which means that nobody is really 'reading' a newspaper anymore.
We also see this with how many news sources people are exposed to. In the past, because we were actually reading a single newspaper, the definition of a newspaper was just that one publication. Today, as we are reading the news via a link, we are exposed to hundreds of different newspapers every week, in which we may only read one or two articles.
When Canada's Vividata says that 81% of the population is 'reached' by newspapers, they are not actually wrong. But that reach has nothing in common with how we defined that reach 20 years ago. It isn't real reach at all.
Reach today is defined in the same way as retail stores measure foot traffic in large malls. Yes, you get a ton of people walking by, but their intent has very little to do with any actual sale, nor is it directed to any specific store. It's a useless metric.
So what does this have to do with misleading media graphs?
Well, let me show you. Here is a graph from QZ, which is just one of hundreds that make this mistake.
What you see the old definition of media, in which each format is sharply defined with no overlap. This study would have made sense in the 1970s, but it's meaningless in 2016.
For one thing, what the heck does 'internet consumption' mean? Is that only the consumption that goes to traditional publishers? Or is that general internet use? Does that include blogs, Facebook, Instagram, Snapchat?... does it include the time you spend in Outlook reading emails?
It's a completely useless definition.
Secondly, when they asked people if they were reading newspapers, did people actually distinguish that as print newspapers, or were they thinking about any exposure they might have had to newspapers in general?
We also have the problem that today's world is multi-device and multi-platform. So people might be watching something on Netflix while using their smartphones. So should that count as two separate activities (by adding minutes to the total), or are both of those really part of the same time (and thus not adding to the total)?
In other words, when QZ claims this study proves that "We now spend more than eight hours a day consuming media", is that actually true? Is it more like we are spending five hours, with three of them using a mix of devices?
Think about what that means in terms of attention, retention, loyalty. There is big difference between a focused period of media consumption (which was what we had in the 1970s), and an unfocused period of media consumption, which is what we see with all the micro-moments people have today.
In the past, people were fully aware of what newspaper they were reading. Today, when you click on a link, you often cannot even remember what site it was on two minutes later.
So, this graph tells you absolutely nothing because it's focusing on the wrong definition of media consumption. It's assuming that people behave the same way today, and that we still live in a divided world of media scarcity as in the 1970s.
It's crap.
What we should be doing instead is to measure how people consume media based on what type of moment they have, and what intent that moment carries with it. The format of media is irrelevant. That's the old world.
In a very simplistic term, we can define this like so:
What you see here are the two major types of moments that we have, micro and macro moments. And within each are many different types of intent. We have those who are merely snacking, those who have a specific need, and those who consume media because of their passions.
Imagine how much better a media study would be if this was how it was measured. Instead of asking 'how much time do you spend reading newspapers?', we would ask 'how much time do you spend snacking on media while also doing something else?', and 'how much time do you spend reading articles about something you need to get an answer to?'
And think about the study from Canada, the one showing that 81% of the population is reached by newspapers. What if we instead measured this in terms of moments. How many people do you reach as a low-intent micro-moment, and how many do you reach as a high-intent macro-moment?
You see the difference?
But no study is doing this today, because they are all based on the old definition of media consumption. Even when the data is right, the result is wrong because they are looking at it the wrong way.
This is why it's so important to stay vigilant and to be a skeptic about the studies that we see. Always question the data. Always question the methodology. Always take that step back and ask if this data really measures what you think it measures.
Creating a propensity model is one of the most important tools publishers can have.
Several publishers have found that reducing volume leads to an increase in revenue
Time is such a critical metric for publishers, but it's also a very complicated one.
Publishers who start their own data studios need to take extra steps to identify real people.
Founder, media analyst, author, and publisher. Follow on Twitter
"Thomas Baekdal is one of Scandinavia's most sought-after experts in the digitization of media companies. He has made himself known for his analysis of how digitization has changed the way we consume media."
Swedish business magazine, Resumé
plus
free
plus
plus
plus
free