Strategic insights
Analyze The Data You Don't Have

You need to consider the data that you don't have, and the things that you don't know. Sometimes it is enough just to know that there is something that you don't know, other times it's necessary to figure how to unlock the hidden data.

If you want to be successful in the new media world, there is one thing that you always need to do before you make a decision. You need to look at the data that you don't have, and analyze the things that you don't know.

Failing to do so, is actually one of the main reasons why most social media studies are completely worthless. They only look at the data they can see, and based on that they draw a conclusion that they have no basis to make.

I know how this sounds. How can you analyze data that you don't have? How can you make a decision based on something you don't know? Isn't that impossible?

Well, no. It is actually much easier than you might think. But before I get to that, let me illustrate the problem.

I was recently watching a talk by Dr Neil DeGrasse Tyson that he gave a couple of years back. It is a long (and funny) talk about all kinds of things, but part of it was about math illiteracy. For instance, when a reporter says that "more than 50% are below average"...Mmmm...

But he gave another example, which I just love because it's very similar to what we see in many new media studies (and infographics). It goes like this: "80 percent of airplane crash survivors had studied the locations of the exit doors upon takeoff"

A survey company has apparently contacted a group of airplane crash survivors and asked them this question, and the result seems to be remarkable. It was 80% of those who survived.

So what conclusions can you make? Many will say that studying the exit locations will improve your chance of surviving a crash, right? That's what people think when they see this data. It was 80% of those who survived!

But then you start to use your brain, and you quickly realize not only is this data completely useless, the study is also a massive waste of time, and should never have be done in the first place.

Why? Because of the data that they don't have, specifically: "What percentage of the people who died had studied the locations of the exit doors?"

We don't have this data, because (sadly) those people are no longer alive, but without knowing the other percentage this entire study is meaningless. What if 86% of the people who died had looked at the exit doors?

Suddenly, the conclusion you formed before is now completely invalid. Because more people had died after studying the exit doors than the people who survived. Or more to the point, wether or not you look at the exit doors has no correlation to your chance of surviving a crash.

I'm just making up numbers here, because we don't know what the percentage is for the people who died. But that is the whole point. If you do not consider the potential impact of the data that you don't have, you have no basis for making a conclusion - and you risk reacting to just what can see and thus make the wrong conclusions.

It is vital to consider what you don't know!

How does this relate to social media?

Let me give you two examples. Let's start of with Klout and Klout Perks. The problem with Klout is that they are *only* looking at the data they can see, and completely neglecting to take into account the data that they can't see.

The data that Klout don't have is the actual conversion rate by each person. Which is a vital data point.

Consider these two people. One has an amazing Klout score of 89, the other has a Klout score of only 52. Which one should you try to influence with your brand?

If you are just reacting to the data, you want to get in touch with that guy with a Klout of 89, but you risk making the wrong decision. You have no idea if that 89 guy is any good at persuading people to buy your product. It might be that he is just really good posting cute pictures and YouTube videos on his Facebook page.

You have to consider the data that you don't have - in this case, the actual conversion rate. And if the data you don't have invalidates whatever data you do have, it is useless.

What if the person with the low Klout score has a conversion rate of 14%, while the conversion rate of the high-ranker is only 2%. As in, the percentage of actual sales caused by each person.

Klout is like the airplane study. It's completely useless because you have no idea if the data is valuable or not. You have to consider the impact of the data that you don't have.

This is a gigantic problem, for instance, when it comes to Klout Perks. Brands will go and say, if you have a Klout score higher than this or that, you get a free Perk. But they have no idea if the people they reach will have real impact.

Don't get me wrong. I'm not saying that Klout Perks couldn't be useful or that people with a high Klout score are worthless. I know several people who have both a high Klout score and at the same time are very good at converting their influence into real value.

What I am saying, is that the determination of whether someone is valuable or not doesn't come from Klout. Value comes from other indicators. Like how many products that person sells? How many people attend the events that this person creates or attends? What is that person about - cat videos or real useful advice relevant to your brand?

But you also have to consider people with a much lower Klout score. I know many people who's Klout score is between 40-50, but who are immensely influential. People who are producing tons of real value for other people and have real influence.

Klout is only looking at the data that it can see, and they are missing the most important data point of all - the conversion rate.

Social ROI

The other example I want to give you is how important "things you don't know" are to prevent you from making the wrong decision in terms of calculating social return on investment.

Let's imagine you are a brand and that you are selling your product in some kind of web shop. One of the ways you can then calculate social media ROI is to make sure that the links you post to your products are unique for each channel. There are many ways you can do this, for instance by adding campaign variables identifying links posted on Facebook as 'Facebook' and links to Twitter as 'Twitter'.

And then, in your analytics, you follow each link, from each channel, through their respective conversion funnels to the point of an actual sale.

It's actually very simple to do.

The result often looks something like this. Facebook seems to be driving the highest amount of sales, followed by Twitter, Google+, Tumblr and maybe Pinterest.

The question is, what conclusion can you now make? What channel should you focus your attention on?

At first glance, you might say, "Oh...well, Facebook of course. It's huge! And Google+ is not really that important". But Facebook is not the biggest channel. If you add them all up, they only amount to a total of 26% of your overall sales (in this example).

The most important channel is the data that you don't know. The 'unknown channel'.

If you just react to the data that you can see, you are basing your decision on a result that neglects to take into account the 74% of your overall sales. That could be the worst decision you have ever made.

What if that 'unknown' block is actually sharing caused by the link you posted on Google+, but it just doesn't show up as such? Of course that would be extreme, but the point is that you can't make a decision to focus on just one channel, if most of your sales are 'unknown'. Just like the meaningless airplane study, you need to know what it is that you don't know *before* you can use it for anything.

As an example of this, I recently studied the power of just one tweet, posted by one person, on one channel. I can do this because of the way this site is designed. Instead of having one link that everyone shares, each subscriber sees a unique link that identifies them in relation to each specific article (just look at the link for this article).

What I discovered was remarkable. Links don't stay on the channel they are posted on. People will reshare them on many other channels. The graph below (from the article 'The Power of a Tweet') illustrates the sources from a single link originally shared on Twitter.

What this means is that, in my case, Twitter is actually 127% more powerful than what my statistics tell me. And about 10% of my Facebook traffic is actually caused by Twitter. That is data that you don't have, unless you look beyond the numbers.

If you can't measure it, test it!

If you cannot do 'some magic' and test your unknown data is some other way (manually or whatever), you can still identify the data that you don't have by doing a series of tests. You need to eliminate the unknown factors.

How easy or hard this is going to be very much depends on your type of product, and their conversion paths. It is hard to do with cars, because people spend a very long time thinking about buying one and car manufacturers rarely make new cars.

But let's imagine that you are a fashion company and you are about to launch a new Spring campaign. Fashion companies are easy, because you have a ton of products with a very short conversion path.

What you do is to find six products that you think have a high potential for sale, the six products that you believe in the most. Then you publish them on your web shop using a hidden URL, that is, a URL that no one can see by just visiting the shop directly.

Then you create your normal social campaign around each of these products, you know, the 5 to 15 posts you plan to make for each. But, instead of posting them on all your channels, you take three of your products and *only* post about them on Twitter, and post three of them *only* on Facebook.

Then you measure the click-through rates for each URL, the conversion funnels and, of course, the actual sale. And by doing this you will know exactly what your return on investment is for each channel.

What you're likely to find is that the data is going to look weird, because the posts that you *only* put on Twitter will suddenly start to show up on Facebook, Google+, Pinterest and other blogs. But that's the magic of social activity. It's not a destination - it doesn't stay where you put it.

But by doing this test, you will know exactly how powerful Twitter is compared to Facebook and vice versa. You will unlock the unknown data.

And you need to do this with every channel you use. But you also need to do multi-channel tests. For instance, posting a video on YouTube and *only* reposting it on Facebook (but not Twitter), then a week later post another video on YouTube, and *only* post it on Twitter (but not Facebook).

Same with blog posts. How much traffic do you get if you don't share it yourself at all (the answer will probably surprise you) compared to how much if you only post it on Twitter?

Note: In my case, the difference is about 300 unique page views (on average), per article, within the first 3 days. But most sharing is done by other people rather than myself. In some cases it doesn't really matter if I tweet something or not. In others, it matters a lot.

You need to consider the data that you don't have, and the things that you don't know. Sometimes it is enough just to know that there is something that you don't know (as in the case of the airplane study), other times it's necessary to figure how to unlock the hidden data.