plus
Everyone is talking about the theory of Facebook EdgeRank, but are their assumptions actually true. We are about to find out!
Facebook EdgeRank has been a hot topic ever since it was created. Some defend it vigorously saying that it brings in added value, while others believe that it takes it away. Then we have the debate about promoting a post to make yourself more visible in people's newsfeeds, and the drop in reach for most brands.
The problem is that most articles only look at a small piece of the puzzle and based on that, people make conclusions about the bigger picture...conclusions that aren't exactly true. The other problem is that many people only look at the theory of how Facebook EdgeRank is supposed to work, and then based on that they engage in a certain way.
For instance, many social consultancies are encouraging you to create highly engaging posts because that will, in theory, make your future posts more visible to more people. For instance, Simply Measured recommend that you post more "Fill in the blanks" posts because they drive the most engagement.
You know, the type of post where a brands asks: "I like hot chocolate because ________", encouraging you to add the missing word (post a comment) which will increase the brand's EdgeRank score (in theory).
But is this true? Is Facebook really that shallow, or is there something else going on here that these social consultancies don't see because they are too blinded by their narrow surveys?
Back in September when most brands suddenly experienced a massive drop in reach, I was getting fed up with all these incomplete studies and erroneous conclusions. So during the entire month of October and most of November, I studied data for three very carefully selected brands to see if I could find the real answer to just how Facebook EdgeRank impacts brands. And the result, detailed in this report, was surprising in many ways.
Here for instance, is how Likes, Comments, and Shares impact how often a post is repeated in the news feed:
Before we look at the data, let's just briefly sum up why EdgeRank even exists. Everyone who is working with social media knows that the true power lies in the news feed. This is true for Twitter, Facebook, Google+, Instagram, Pinterest, LinkedIn, and all the others.
There is just one problem. All of us are following more people and brands that we can possibly keep track of. For instance, I only follow about 225 people on Twitter, but combined they produce about 180 tweets per hour. If I only check my stream once per hour, seeing about 10-20 tweets, I actually only see about 8% of my stream.
Of course, I can always go back to see more tweets if I wanted to, but none of us have time to see everything. This is true for every social channel. We don't see every update, even if we could.
The problem with Twitter's approach is that, because it's unfiltered, the value of the tweets that we do see is not always very high. You might not see that really exciting and influential post, simple because it was posted during the gray zone when you didn't check your stream. At the same time you might see somewhat boring updates, because those happen to be what people posted just before you checked.
This is a common problem, and it is why several news aggregators like Zite and Flipboard, offer to reorder your stream based on how important each post is.
Facebook is faced with the same problem, and as result they invented EdgeRank. In theory it's very simple. Instead of showing you an unfiltered stream, Facebook will rank the relevance of each post and show you an edited newsfeed instead.
In theory this is a good idea, except that social media is not about editing. We don't engage socially to have someone else edit our connections. We engage socially to have a direct connection to what and whom we care about.
And the problem is that it's based not on what we want (influence), but on what we do (engagement). This is why we see so many studies that tell us that "fill in the blanks" are better than posting something about your latest product.
Google+ also faced the same problem. But instead of choosing one or the other, Google decided to do both. When you add a brand to a circle you get to decide how much or how little you want to see from it.
You can create a circle where you tell Google to show you everything, unfiltered. Or you can say, only show me what you think is important.
This is good because it allows you to be in complete control of your social connection. But the downside is that it's a lot more work, and most people will never change their circle preferences.
In reality, neither solution is optimal and we are unlikely to ever find one that is. The problem with Facebook of course, is that it is the only platform that has completely taken away our control, and that creates a lot of friction.
So, what about all this talk about brands experiencing a massive drop in reach, around the same time that Facebook launched promoted posts? Surely something screwy is going on here, right?
Well, yes and no. Something is definitely going on, but not in the way most people think (and surely not the way it's being reported by most of the media).
EdgeRank is based on four things:
And the fourth element is a new thing that really came into play in late August.
In short, engagement and the type of post improves your EdgeRank score, while the time decay and the negative feedback makes it worse. These four factors combined is what determines the success of your post.
So how does this result with our lower reach and the introduction of promoted posts? Well, here is the thing:
As I illustrated above, we never see every post. So if we assume that people only see 10 posts every time they check their stream, this is what will happen:
First, promoted and sponsored posts are going to show up for about 20% of the posts. We may not like that, but this is how Facebook makes money (and the only viable way to make money on mobile devices).
This alone reduces reach for organic views by 20%. So yes, promoted posts have had a big impact on reach (just not specifically because of EdgeRank).
What's left are 8 spots that Facebook will fill based on the four EdgeRank criteria. In short, there are three types of posts of interest to us here.
And each of these have to compete for the remaining 8 spots. So how does that work? Well, the best visual example comes from EdgeRank Checker who, for a very short time, was able to see their news feed both with and without EdgeRank (which Facebook quickly shut down).
It very clearly shows us what's going on:
When EdgeRank is applied, updates from brand pages dropped 61%, friend posts increased by 33%, and activity updates (friend actions) dropped by 81%.
All of this makes a lot of sense. Since brand pages will always experience a higher rate of negative feedback than posts from friends, it's no surprise that they have a lower EdgeRank score.
The real surprise is what happened to the activity updates (friend actions). When we hear about these most people assume they are posts about what people are doing in Farmville, songs they are playing in Spotify, or newspaper articles read on the Guardian app.
And yes, those are all in that group.
But it is also activity about what your friends are doing on the brand pages people follow. For instance, "Thomas posted a comment on Porsche's page" is an activity update (friend action).
And these type of updates are now filtered out 81% of the time (according to that study).
Remember back when all social media consultants told brands that it was very important to get people to like and comment on a brand post, because when they did, all their friends would see it too?
Not anymore! That is an activity update (friend actions).
Liking and commenting is still important because it increases visibility in the newsfeed (as you will see below), but it no longer contributes to generating additional reach towards friends of fans. That's a huge difference from the past.
So it's the combination of the following three things that is causing the drop in reach all brands have experienced these past couple of months:
When Facebook says that reach is the same overall, it is most likely telling the truth. It's just not a truth that benefits brands in any way.
But enough about the theory, let's look at the data so that we can see exactly what EdgeRank does and how it affects us.
For the past one and a half months, I measured data for three brands, in order to see exactly how EdgeRank worked. The way I did this was to create a new Facebook account, follow these three brands, and every day record where in the newsfeed each brand post was displayed. The top post got a score of one, while the 10th post got a score of zero.
The three brands are:
Each of these has the same publication schedule, posting about one post per day. This means that in theory there is no need for Facebook to apply EdgeRank or filter the stream in any way. With only three posts per day, I should always be able to see the three latest posts.
The test was divided into three stages:
Here is what it looked like in its RAW form. The darker the color, the higher up the post was placed in the newsfeed.
If this was Twitter, then each day would have a completely granular view. With the newest post showing up in dark green, and then gradually moving into light green for each post after that. But as you can see, while every post (except one) actually showed in my stream, the order of how they appeared varied greatly.
You can see one example of this in the picture below. What we see here is that on October 15th (top/down), the latest post showed up at the very top of the newsfeed. But the 2nd and 3rd newest post was filtered to the very bottom of the stream (9th and 10th place). Instead several older posts, some more than a week old, were pushed to the top.
For brands this has a huge impact, because it is much less likely that people will continue to scroll down, if they see something they have seen before. But it's also interesting, because it means that older posts have a chance of renewing themselves and gaining additional exposure.
We see this effect far more clearly when we look at the last stages of the test (as I followed six more brands). Now we see how EdgeRank is not only ranking the newest posts, but also finding older posts and repeating them about four days after they have been posted.
So again, we see the dilemma here. On Twitter you only get one chance to influence people. On Facebook, your exposure might be reduced initially, but as engagement increases your post might show up more than once on the following days.
But let's look at it in a more simplified way.
Remember that the test involved three stages, first following three brands, then following six more, and finally engaging with one brand (Porsche) to see if I could skew EdgeRank in it's favor.
We would expect that when liking three times as many brands, the view score would also drop to 1/3 of what it was. But here is what actually happened:
While the View score did drop, it didn't drop as much as one would expect. Meaning that EdgeRank favored brands I had followed longer than those I just added recently.
I did not expect to see that. I would have assumed that because I hadn't engaged with any of the three original brands, the view score should have dropped far below the red line. So it might seem that the time element in EdgeRank actually works both ways. It reduces views for older posts, but increases the EdgeRank the longer you have liked a brand.
Also notice how the final part of the test increased the view score (each post ranked higher in the newsfeed), but for fewer days.
In fact, when looking at it per brand, we see no clear indication that engaging with Porsche made any difference. Although, at this level, the fluctuations in the data exceed the variance in the data set (i.e. not statistically significant).
You will notice in the above graph that Avinash's lines are longer than those of Porsche and Nieman Journalism Lab. Does that mean that Avinash has the best performing brand page?
When I started this test, I very carefully chose these brands because of the three different ways they use Facebook. I expected Porsche to be the clear winner, because they are doing all the things that social media consultants encourage brands to do (and they have a very popular brand).
I expected Nieman Journalism Lab to perform poorly, because they only post links. And I expected Avinash to come somewhere in between, because his posts are more blog-like than the quick snacks we are used to on Facebook.
Here is one example of a Facebook post from Avinash. It's not exactly the type of post most social media consultants would tell you to make. It's long and there are no pictures of cute cats, or social surveys, or posts saying: "like this if you drink tea, comment if you drink coffee, share if you just want a beer".
One example of 'advice' that you are often given by social media consultants is this:
Keep it short and to the point. Posts of 0 to 80 characters have a 27% higher engagement rate.
So Avinash's posts shouldn't work on Facebook according to what these 'expert studies' tell us. But they do.
Overall, Avinash performed far better than both Porsche and Nieman Journalism Lab. Here is a combination of their view scores. Avinash performed 77% better than Nieman Journalism Lab, and 22% better than Porsche.
When we look at how many days Avinash's posts were visible, we see the same pattern:
And when we see all the posts visualized together, we clearly see that Avinash (orange) is dominating, while Nieman Lab (red) is lagging behind, and Porsche (blue) is scattered all over the place.
And it's the same when we look at how many days a post was visible:
Isn't it curious that the best performing brand is one that focused on creating long term value, rather than short term bursts of engagement? Obviously, this also has a lot to do with the kind of audience that Avinash has, but Avinash doesn't focus on engagement. He focuses on being influential.
We would assume he would be punished for that by EdgeRank, but he isn't. In fact, themost liked post of all from Porsche received 33,000+ likes, but only scored 0.7 in View rate. Theworst performing post of Avinash's (with only 28 likes) scored 1.6. That's 128% better.
We also see this when we look at the average View rate per day. Now Porsche scores higher than Avinash (more engagement). But remember that Avinash's posts were visible in the newsfeed for an average of 4.7 days, while Porsches' posts were only visible for 3.6 days.
So because Porsche has more engagement, they experience a bigger initial boost (better in the short term), but over time, Avinash wins because of a higher overall level of exposure (better in the long term).
EdgeRank is a tricky beast. Just getting a lot of people to engage is not all there is to it.
There is, however, a catch. During this test I assumed that every person would look at 10 posts every time they visited Facebook, and that was what I measured. I gave the top post a score of one, and the 10th post a score of zero.
But what if that assumption isn't true?
Think of it like this. When you go to Facebook and you see a post that you've already seen before, do you continue scrolling? Or do you stop assuming that you there are no new updates for you to see?
In the graph above I assumed that people kept scrolling. But what would happen if people didn't? Well, Porsche suddenly takes the lead. In the graph below, we see the View scores where we assume people stop scrolling the moment they see something they've seen before.
I don't have any data to support either assumption. Just for fun I asked my followers about what they did and 82% said they stopped scrolling when they saw an old post (but my followers are hardly 'normal' because they are all media professionals).
If we look at it on a per post basis, things are less clear. It is a lot less mixed than before. In fact it seems like Porsche's lead is caused by a few high performing posts.
Are you seeing the right picture, or is our perception being skewed by a few posts? The simplest way to answer that is by looking at the median score, instead of the average.
And by doing this we see a slightly different picture. Avinash is better when we just look at EdgeRank, while both Porsche and Avinash match each other's performance if we assume that people stop scrolling when they see something they have seen before.
Another factor is the overall visibility. If we assume that everyone scrolls down 10 posts, then all of Avinash's and Porsche's posts were displayed in the feed. And 26 out of 27 of Nieman Journalism Lab's post were also displayed.
However, if people stop scrolling, that number drops by about 10%. Instead of having almost all the posts appear in the stream, now only about 90% of them are visible.
And remember, during this test, I only followed these three brands. Imagine what the effect would be if you followed 30 brands, or 100. Very quickly you end up in a situation where only a small fraction of a brand's posts are ever seen.
Because EdgeRank doesn't show posts in chronological order, how people scroll is a very big deal.
Let's talk about how important EdgeRank actually is for how a post is displayed in the news feed. We all know the theory. Brands need to get people to engage, otherwise their posts won't show up in people's feeds.
Is that actually true, and to what extent is it true? Well, let's look at Avinash's posts.
Let's start by looking at Likes, Comments and Shares in relation to how each post appeared in the news feed. As you can clearly see, when the number of Likes, Comments, and Shares goes up, so does the view score.
Engagement is indeed very important.
We see the same pattern when we look at how many days each post was visible for. When the number of likes, comments and shares goes up, so does the number of days the post is repeated in the news feed.
So far we have only looked at Avinash, but how did the graphs look like for Nieman Journalism Lab and Porsche?
Comparing the View rate with engagement for Nieman Journalism Lab shows the same pattern, although to a lesser extent.
And it's the same when we look at days visible:
The problem with Nieman Journalism Lab is that because they are only posting links, their engagement is very low. Many posts have zero comments.
If we instead look at negative feedback, which for Nieman Journalism Lab is very low, we again don't really see any correlation.
For Porsche we have the opposite problem. Their massive scale also increases the range of variance. We still see a correlation, it's just not as clear as with Avinash.
And again, it's the same when we look at the number of days a post is visible:
Notice however, how much more closely the rate of Likes, Comments and Shares follow each one on Porsche's page. When Likes go up, so do the other forms of engagement, and vice versa.
I'm not really surprised by that, but it's still interesting to see.
But remember, Facebook EdgeRank is no longer just about Engagement. Since September it's started adding negative feedback into the mix as well. So how does that correlate with how something is displayed in the newsfeed?
Well, that is not as clear.
What I have done here is to flip the negative feedback curve, so we can better identify a possible correlation between the View score and the level of negative feedback for Avinash's posts. It's not obvious that there is an overall correlation.
I also tried putting the data on the same scale, with negative feedback dragging EdgeRank down, while Engagement increased it. For us to see a correlation, we should see some kind of pattern in the View score (black bars). But again, I just don't see it.
We see the same picture when we look at Nieman Journalism Lab. There doesn't seem to be any correlation between how often it is visible, compared with the level of negative feedback.
We know that EdgeRank is affected by the level of negative feedback. Facebook even invited the press to tell us about it. They said:
If a specific post has received complaints by other users who have seen it, or the Page who posted it has received lots complaints in the past, you'll be less likely to see that post.
Facebook also told Techcrunch that:
A news feed algorithm change did start reducing reach for Pages that get complaints and that successfully cut total complaints by a double digit percentage.
I can't see this from any of the data, but that might be because both Nieman Journalism Lab's and Avinash's negative feedback rate is so low. I think the main cause of a drop in reach (which I described in the beginning of this report as the lower exposure of activity), is a shift towards friend posts and more space taken up by promoted posts. I have no doubt that negative feedback does play a role, why else would Facebook invite the press to point it out?
So what is negative feedback exactly? Well, it's every time one of your fans reports that your post is something they don't want to see. This can happen in three different ways:
So compare these two posts:
One is of the traditional high-engagement type, where you encourage people to like a picture or message. The type that most social media consultants tell you to post. The other is an insightful, post from Avinash.
Which one would perform the best?
I don't have data specifically for these posts, but I would bet that Avinash's post would perform better than the one from Costa Coffee.
Ask yourself this: "How many people do you think would provide negative feedback to the one from Costa Coffee (because it is basically just wasting people's time), compared with the level of negative feedback for Avinash's post."
I don't know the number from Costa Coffee, but I can tell you that only one person reported Avinash's post (by choosing to hide it), or only 0.03% of that post's audience.
So I couldn't identify a correlation between each post's View rate and its level of negative feedback in this study. However, you need to be very careful about how you drive engagement . based on what Facebook is telling us.
Also, to get increased reach, you need to get people to share your post. Just liking or commenting on it no longer causes your friends of fans to see it to the same extent as before. Reach is now the result of sharing, not liking.
Finally, we need to discuss conversions. One thing I keep telling brands is that I don't care about their level of engagement, I only care about how much money they earn.
Obviously it's better to have a brand page with 100,000 followers than one with only 5,000. But that doesn't mean it's better to have a post with 2,000 likes than one with only 500.
Of course, it depends on the type. So let me give you two examples.
If you are posting two different products, and one of them has 350% more likes, as in the example below, it's probably best to assume that the dress sold more than the pink t-shirt.
On the other hand, if you have one post with 2,000 Likes, encouraging people to like some random message, compared with a product post with only 500 likes, the post with more likes probably didn't actually sell more. I would be surprised if it even sold one.
So measuring conversion is critical, because there is often no correlation between what people engage with, and what causes them to buy a product.
Now, I have to admit that conversion is bit tricky for the three brands in this study. Avinash, in his role as a Digital Evangelist working for Google, doesn't sell a product. Instead his success is based on how influential he is. That's very hard to measure on Facebook on a per post basis.
But based on the result illustrated earlier, it's safe to say that he is successful at what does. His brand page is above, or matching, Porsche in reaching his audience.
We can look at clicks and what we see is something very interesting:
First of all, you will notice that there seems to be no correlation between how many people click on a link, and the post's View rate. In fact, one might even say that there is an opposite correlation, because almost every time a post receives a high number of clicks, the View rate goes down.
One might ask if Facebook is punishing brands from linking outside their site? It sure seems odd. Of course, it might be caused by other things. If we compare this to the engagement level, every time a post contained a link that people had to click on, the engagement level dropped. And as we know, a lower level of engagement equals a lower view rate.
The other thing you will notice is that all the links that performed well, were videos posts with clips from either YouTube or Vimeo. And the highest performing clicks were to the video interviews featuring Avinash.
That is a conversion!
However, the one link to Avinash's blog performed very badly. That is a problem, because his blog posts are very indepth and something he puts a lot energy into (and is very popular on its own).
I can't really tell if this is true problem, because with only one post there simply isn't enough data to work with.
If we instead look at Nieman Journalism Lab, like Avinash they are also in the business of selling 'influence', by getting people to read their articles on their site. As such, clicks go a good way to measuring their success.
And if we look at the data, we see a different picture:
On one hand, it looks like engagement is really the cause behind the View score, and that the number of clicks is merely the result of people seeing the posts.
But then we have a few odd posts that makes this a bit more complicated. Look at the area highlighted by the grey circle. Those are two different posts, and one has a high click-rate but a low engagement score, while the other has a high engagement score, but a low click-rate. Yet both of them have a high View score. The same thing happens a few posts later.
Does that mean that the more people that click on a link to Nieman's site, the more likely it is to be viewed by other people in the newsfeed? Well...maybe. The data is not conclusive.
Measuring conversion rates for Porsche, of course, is even trickier. They do sell several products (their cars and accessories), and many people buy them. But nobody buys a Porsche just because of a single post on Facebook.
Instead Porsche needs to measure how much people desire to drive a Porsche. Because it's this desire that eventually causes them to save up enough money to buy a Porsche.
One thing I haven't covered in this study is Promoted Posts, except how it accounts for about 20% of the newsfeed (and therefore causes a drop in overall reach).
And the reason I'm not focusing on it, is simple I don't have enough data. Also, it's because the data and the things I hear about Promoted Posts is really weird (and not in a good way). We hear several people complaining that promoted posts seem to be targeted at the wrong people.
But overall, I have yet to see a single example where a Promoted post acts in the way that you expect it to. I'm not going to go into this in detail here, but let me just end this report with a simple example.
Avinash was generous enough to promote four posts during my study, and I was anticipating some kind of effect. Here is what happened instead:
First, if we look at the total reach reported by Facebook, we clearly see that promoted posts reach far more people. But the promoted post didn't show up in any different way than non-promoted posts.
But here is the weird part. If we look at Reach specifically in the newsfeed and on Avinash's page, Facebook reports that about the same people saw it in the newsfeed and on the page.
This is data as defined by Facebook as: "The number of people who saw your Page post in News Feed or ticker, or on your Page's Wall."
Aren't promoted posts supposed to increase the overall visibility in the newsfeed? Isn't that what we are paying for?
And if we look at the number of fans who saw the posts, we see the same strange behavior. Avinash didn't actually reach more of his fans when promoting a post. Instead it kind of looks like it's just targeting people who would have seen the post anyway.
So if a promoted post does *not* increase Reach via the newsfeed, your page, or for your fans... who the heck sees it? ... and how? Something is wrong here, and I honestly don't know what.
As I said, I don't have enough data to provide an answer to what promoted posts actually do. All I have is a huge number of very conflicting reports and data, that all point to the fact that promoted posts don't behave in any way how we would expect them to.
So if you test promoted posts, make sure you measure it. Don't just look at the data that Facebook gives you next to you post. Look at the actual RAW insights data, and measure it against a real outcome (like channeling people to a product, a blog post, or getting a sale).
I hope this report is useful in providing you with very indepth look at what actually goes on in Facebook, and how EdgeRank impacts your brand.
EdgeRank is changing all the time, but the direction in which it's changing is very clear. Facebook will continue to focus on finding the most engaging content across both personal and brand pages, while punishing those that people either don't engage with, or react negatively towards.
At the same time, EdgeRank pretty much works like an editor, deciding what people can or cannot see. You can promote a post, but as of yet, it isn't clear that this solves anything.
Instead, look at Porsche and Avinash for inspiration, and don't listen to the many social media pundits who tell you that "fill in the _____" or "click like if you..." is the best thing to do.
Engagement is important, but it has to be the inspiring kind.
--
Head over to G+ to comment and discuss this report
Creating a propensity model is one of the most important tools publishers can have.
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Several publishers have found that reducing volume leads to an increase in revenue
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Time is such a critical metric for publishers, but it's also a very complicated one.
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Free for subscribers
...or full access for $12
Publishers who start their own data studios need to take extra steps to identify real people.
Free for subscribers
...or full access for $12
Founder, media analyst, author, and publisher. Follow on Twitter
"Thomas Baekdal is one of Scandinavia's most sought-after experts in the digitization of media companies. He has made himself known for his analysis of how digitization has changed the way we consume media."
Swedish business magazine, Resumé
plus
free
plus
plus
plus
free