Watching acquaintances raise funding around metrics like “viewership”, “engagement”, “reach”, and “accounts” has made me especially weary of companies espousing some kind of Golden Metric (big red flag if they don’t mention earnings/money) as a means to justify they are an awesome business.
It’s just too easy to “fudge the numbers” and give the very-best-case estimates that don’t incorporate uncertainty or sensitivity (or reflect reality). It just takes some non-half-assed due diligence to do a sanity check.
> ”We had 100k new users signup in January”
> “How many were still active a month later?”
> ”... 17k”
...
> ”Our users read an average of 12 stories per day”
> “What counts as a ‘read’?”
> ”Spending at least 3 seconds on a page”
It never gets old the stretches people will make to try to pitch something that is in desperate need of legs to stand on, so whatever metric that looks the best is the one pulled out of the hat.
Ideally you want to correlate the metric with the dollar ROI. I have yet to see extensive reports on that about google & FB. It's crazy to think the amount of money being thrown at them willy-nilly
I'll be playing the devil's advocate because there is a marketing department down the corridor and we get along (mostly) fine:
>> ”We had 100k new users signup in January”
>> “How many were still active a month later?”
>> ”... 17k”
> ...
Well, that's a retention rate and it's a useful measure (like engagement and reach are).
>> ”Our users read an average of 12 stories per day”
>> “What counts as a ‘read’?”
>> ”Spending at least 3 seconds on a page”
3 seconds might be more than you need to get your message on your audience's mind though. I am thinking of the catchy headline, the picture of the celebrity of the month (with that particular brand of headphones or handbag), the coke ad, etc. That 45second of vid. that isn't played ? The first still is the message.
Now from that, how do you go to prove that your conversion rate is tied to those measures ? I don't know. The consensus seems to be "we can't really pinpoint it but when we don't do it our conversion rate tanks and we get less sales/booking". Yeah, marketing isn't as clear cut as quadratic equation. In a way, it's not that easy.
> It never gets old the stretches people will make to try to pitch something that is in desperate need of legs to stand on, so whatever metric that looks the best is the one pulled out of the hat.
Which is fair game. Contract, commercial endeavor, etc. Information is always asymmetrical.
Now, in my 9 to 5, I have had that advice since day one: your $50 FB/GA campaign for a year or a month isn't going to fix a bad business plan :/. If your product is flawed, get out before losing any more money.
> [be] especially weary of companies espousing some kind of Golden Metric (big red flag if they don’t mention earnings/money) as a means to justify they are an awesome business
Solid, sound and gold advice. Most agencies are working on the basis of "best efforts" but it pays to ask them for success stories and check them out.
Now, what facebook did ? Pure, evil scam but I wouldn't hold my breath to see if anything changes in "the industry" (it hasn't when it was revealed and yet tomorrow I am attending a video festival dedicated to the tourism industry).
My business partner and I did our best to avoid getting caught up in this race to the bottom with our product. He was convinced we needed more followers on FB and Insta and wanted to target our effort there. We did. We tried and the figures just do not support it.
From all the figures we had available (sample size of 1 here!) we worked out that for every 8 sales we needed 10,000 likes of our product.
10,000 likes to get approximately 8 sales.
Not sure what the ratio of views:likes is but I bet it's 100:1 or something.
It's a full time job to manage people on social media but it's a total con: it takes masses of "engagement" (or whatever their bullshit is called) in order to get a sale.
I suppose it's the myth of the long tail marketplace: There's not enough business from it to sustain me but when you scale up to 100 Million business like me (none of which make much money) it becomes profitable for the marketplace owner.
Impression fraud is going to be a huge scandal. So many view counts are inflated for profit - either by platforms themselves or by third-party bots interacting with content.
Once upon a time we have launched a rich media banner with three videos in it. What was interesting that all three videos generated around the same viewability stats. What was event more interesting that one video in banner was hidden with display:none.
I wonder if anybody does test like this - e.g. checks if hidden video generates views, button with visibility: hidden generates clicks and etc. and in this way evaluates traffic.
This is incredibly serious, as this kind of fraud might have lured billions in advertising money to go to facebook instead of other platforms. It's a pity it was settled and we ll not know the extent of the fraud.
It really depends on platform. Adwords/FB are PPC. But many of the smaller alternate display networks charge by CPM (1000 views) or similar.
As a rule of thumb paying for views means lower value. Often networks try and justify it and say we are as 'premium network'. I treat those words as a red flag more than CPM as I'm yet to test a 'premium network' that has better ROI or customers than others. All that said, sometimes CPM can strike gold if you know what/where/who to buy, but if your taking sales pitches from reps its more often not good.
But sometimes your company is pushing for growth, you've maxed out the performing channels and you have to go for the low or negative ROI sources to reach total targets and when all things are merged metrics work out for the business.
Strange, nowhere in the article does it say how did they know that the views where inflated by 150 to 900%. How did they even know, without Facebook admitting it?
I don't think they did, because even if they settled for 40M they say the process is without merit. Where does that information come from?
> Strange, nowhere in the article does it say how did they know that the views where inflated by 150 to 900%. How did they even know, without Facebook admitting it?
In September 2016, the Wall Street Journal revealed that, for the past two years, Facebook had been overstating the average time its users spent watching paid video advertisements. Based on information from advertising agencies who had spoken with Facebook, the Wall Street Journal reported that Facebook’s metrics had been overstated by between 60 and 80%.
In response to the media attention, Facebook admitted it made a mistake, but emphasized that it had only discovered the mistake “about a month ago,”
…
Internal records recently produced in this litigation suggest,
…
Facebook did not discover its mistake one month before its public announcement. Facebook engineers knew for over a year, and multiple advertisers had reported aberrant results caused by the miscalculation (such as 100% average watch times for their video ads).
It seems that advertisers saw things that were obviously incorrect (e.g., 100% average watch times). Some kind of weirdness eventually led to a lawsuit. During the suit, the plaintiffs found evidence in internal documents that indicated that at least some engineers in Facebook knew there was a problem and maybe knew the magnitude of the problem. I wouldn't call that "Facebook admitting it". It sounds more like Facebook handed over some documents that they were legally obligated to hand over and then some lawyers read the documents.
If you're curious about the evidence, most of the court documents are public and available online, although the smoking gun on the percentage appears to be sealed.
How is that even a problem? I ran facebook ads for a few month and I didn't event care about the number of views. All I stared at was the ROI. Ads are profitable ? => I keep the ad runing. Not profitable ? => I stop them.
If some user are stupid enough to run unprofitable ads, it's their problem.
siruncledrew|6 years ago
It’s just too easy to “fudge the numbers” and give the very-best-case estimates that don’t incorporate uncertainty or sensitivity (or reflect reality). It just takes some non-half-assed due diligence to do a sanity check.
> ”We had 100k new users signup in January”
> “How many were still active a month later?”
> ”... 17k”
...
> ”Our users read an average of 12 stories per day”
> “What counts as a ‘read’?”
> ”Spending at least 3 seconds on a page”
It never gets old the stretches people will make to try to pitch something that is in desperate need of legs to stand on, so whatever metric that looks the best is the one pulled out of the hat.
buboard|6 years ago
johnchristopher|6 years ago
>> ”We had 100k new users signup in January”
>> “How many were still active a month later?”
>> ”... 17k”
> ...
Well, that's a retention rate and it's a useful measure (like engagement and reach are).
>> ”Our users read an average of 12 stories per day”
>> “What counts as a ‘read’?”
>> ”Spending at least 3 seconds on a page”
3 seconds might be more than you need to get your message on your audience's mind though. I am thinking of the catchy headline, the picture of the celebrity of the month (with that particular brand of headphones or handbag), the coke ad, etc. That 45second of vid. that isn't played ? The first still is the message.
Now from that, how do you go to prove that your conversion rate is tied to those measures ? I don't know. The consensus seems to be "we can't really pinpoint it but when we don't do it our conversion rate tanks and we get less sales/booking". Yeah, marketing isn't as clear cut as quadratic equation. In a way, it's not that easy.
edit: I was reading this article the other day https://www.thinkwithgoogle.com/marketing-resources/micro-mo... and it mentions steps taken by a traveler https://storage.googleapis.com/twg-content/original_images/l... before an hypothetical conversion. Few agencies have the horse power to analyze and prove those steps are taken for the products they help their clients to sell. But I am convinced it's doable, not in the price range of cheap agencies that boast only facebook likes though. /edit
> It never gets old the stretches people will make to try to pitch something that is in desperate need of legs to stand on, so whatever metric that looks the best is the one pulled out of the hat.
Which is fair game. Contract, commercial endeavor, etc. Information is always asymmetrical.
Now, in my 9 to 5, I have had that advice since day one: your $50 FB/GA campaign for a year or a month isn't going to fix a bad business plan :/. If your product is flawed, get out before losing any more money.
> [be] especially weary of companies espousing some kind of Golden Metric (big red flag if they don’t mention earnings/money) as a means to justify they are an awesome business
Solid, sound and gold advice. Most agencies are working on the basis of "best efforts" but it pays to ask them for success stories and check them out.
Now, what facebook did ? Pure, evil scam but I wouldn't hold my breath to see if anything changes in "the industry" (it hasn't when it was revealed and yet tomorrow I am attending a video festival dedicated to the tourism industry).
_Understated_|6 years ago
From all the figures we had available (sample size of 1 here!) we worked out that for every 8 sales we needed 10,000 likes of our product.
10,000 likes to get approximately 8 sales.
Not sure what the ratio of views:likes is but I bet it's 100:1 or something.
It's a full time job to manage people on social media but it's a total con: it takes masses of "engagement" (or whatever their bullshit is called) in order to get a sale.
I suppose it's the myth of the long tail marketplace: There's not enough business from it to sustain me but when you scale up to 100 Million business like me (none of which make much money) it becomes profitable for the marketplace owner.
dang|6 years ago
Submitted title was "Facebook's viewership metrics were inflated by 150 to 900%".
tomp|6 years ago
cwkoss|6 years ago
AndrewThrowaway|6 years ago
I wonder if anybody does test like this - e.g. checks if hidden video generates views, button with visibility: hidden generates clicks and etc. and in this way evaluates traffic.
buboard|6 years ago
dannyr|6 years ago
It's definitely worth it for Facebook to cheat and break laws when the fine you'll get is just peanuts.
Animats|6 years ago
Gustomaximus|6 years ago
As a rule of thumb paying for views means lower value. Often networks try and justify it and say we are as 'premium network'. I treat those words as a red flag more than CPM as I'm yet to test a 'premium network' that has better ROI or customers than others. All that said, sometimes CPM can strike gold if you know what/where/who to buy, but if your taking sales pitches from reps its more often not good.
But sometimes your company is pushing for growth, you've maxed out the performing channels and you have to go for the low or negative ROI sources to reach total targets and when all things are merged metrics work out for the business.
ing33k|6 years ago
vfc1|6 years ago
I don't think they did, because even if they settled for 40M they say the process is without merit. Where does that information come from?
luu|6 years ago
> I don't think they did
From the complaint at https://assets.documentcloud.org/documents/5004295/d5cb8373-...:
In September 2016, the Wall Street Journal revealed that, for the past two years, Facebook had been overstating the average time its users spent watching paid video advertisements. Based on information from advertising agencies who had spoken with Facebook, the Wall Street Journal reported that Facebook’s metrics had been overstated by between 60 and 80%. In response to the media attention, Facebook admitted it made a mistake, but emphasized that it had only discovered the mistake “about a month ago,” … Internal records recently produced in this litigation suggest, … Facebook did not discover its mistake one month before its public announcement. Facebook engineers knew for over a year, and multiple advertisers had reported aberrant results caused by the miscalculation (such as 100% average watch times for their video ads).
It seems that advertisers saw things that were obviously incorrect (e.g., 100% average watch times). Some kind of weirdness eventually led to a lawsuit. During the suit, the plaintiffs found evidence in internal documents that indicated that at least some engineers in Facebook knew there was a problem and maybe knew the magnitude of the problem. I wouldn't call that "Facebook admitting it". It sounds more like Facebook handed over some documents that they were legally obligated to hand over and then some lawyers read the documents.
If you're curious about the evidence, most of the court documents are public and available online, although the smoking gun on the percentage appears to be sealed.
geocar|6 years ago
http://mediaratingcouncil.org/
Mathnerd314|6 years ago
jonny383|6 years ago
Just in case you were not already aware, Facebook is not to be trusted under any circumstances.
carokann|6 years ago
otalp|6 years ago
theqult|6 years ago
I'm curious to understand how they proved it
gbersac|6 years ago
If some user are stupid enough to run unprofitable ads, it's their problem.
alwayseasy|6 years ago
How do you stare at the ROI for a CPG brand looking to build awareness for their campaign or product?
Media companies that pivoted to Facebook videos looked at user engagement in the form of video views and would market their size to advertisers.