This is essentially what happened in my group when I was at Yahoo. Our group spent several months on a redesign that made it significantly more modern and easier to use. After it went live, though, this decreased the number of ad clicks by a significant number, and the people in charge hurriedly reverted all the changes back, since they needed to make their revenue numbers in order to make their bonus.
When you are stuck in a company that can't innovate because a shitty site leads to more money due to inertia, then you know you are on your way down. This leads to your best developers thinking "what the fuck did I waste all my time for?" and they will leave in no uncertain terms.
I've actually faced a similar situation, and let me tell you it was hard. Companies like Yahoo exist to make money - or, at least, they must continue to make money in order to continue existing. Like Dustin said,
> "This is truly a nightmare scenario for any CEO: do you take the risk and proceed with the better user experience/product at the expense of short term numbers–with no promise that the better design will actually lead to long-term benefits–or do you scrap the new design and start over?"
Would the simplified, modern design have led Yahoo into a new era of profitability? Maybe. Or maybe they'd just have a nicer site that made less money for a year.
Our team made a site that made less per visitor, but we felt like it was a better-looking and more useful site. We decided to stick with it and work out the kinks, but ~6 months went by without reaching our old sites' benchmark. There was a ton of pressure on me to make it work or admit defeat and revert. I nearly gave in as the stress built, but eventually revenue surpassed the old site. I later learned that I was almost fired over it.
My story has a happy ending, but just barely. I have sympathy for the people making these decisions.
Nearly everyone has bought into the utter bullshit that advertising makes the web free. This delusion buries not only the fact that we have made a deal with the devil, but also that the deal really sucks. What we traded our souls for we don’t even get. The web would be both cheaper and better if we just paid for what we use straight up. And more importantly, society would be better. I'll explain all of these.
IT'S NOT FREE
We’re not Facebook’s customers, advertisers are (more on this below). But we are the advertiser’s customers, and the cost of the "free lunch" is simply shifted to the price of the things we buy from them. In other words we still end up paying for the full cost of Facebook (and even more, as I'll get to next). Costs may even shift regressively, to advertised products predominately consumed by those with lower incomes, in which case the poor are subsidizing the better off.
IT'S MORE EXPENSIVE...
Not only are you still paying for the full cost of the Facebook product you use, you are paying for all the advertising overhead: the costs of its advertising technology and infrastructure (huge, btw), the agency and creative costs (Don Draper and company have to pay for the hookers and scotch somehow, not to mention what’s-his-name who basically just lounges in his office barefoot thinking Japanese), and the advertiser's big marketing departments (that often outnumber and outspend the people making the product!).
The best minds of my generation are thinking about how to make people click ads. That sucks. – Jeff Hammerbacher, fmr. Manager of Facebook Data Team, founder of Cloudera
So in addition to the original product cost and the ad overhead costs, you are also paying the opportunity cost of an inferior product (as Dennis Curtis points out in the OP) as well as the engineering costs of figuring out how to optimize ad revenue, because that’s what happens when websites have to design to please advertisers over pleasing us, the users. Dalton Caldwell makes this point comparing Sourceforge to Github[1]. As ergo says in a comment[2], “If the new news feed is making their advertisers happy (and bringing revenue into Facebook), then that's what they optimize for.” As jfoster says in a comment[3], “Ad-supported models untie the relationship between UX and revenue.”
Furthermore, our identities and privacy are bought and sold to the highest bidders. And where do the bidders get their money? From us of course! A double whammy! We're trading our privacy for free product? Bullshit. We get personalization? Bullshit. Personalization means optimizing something for me, not optimizing for the advertiser. Again, we're not the real customer. We’re certainly not Google’s[4][5].
WAIT. IT'S EVEN WORSE...
Advertising has us chasing cars and clothes, working jobs we hate so we can buy shit we don’t need. – Tyler Durden, Fight Club
Think of the social costs of advertising. The web is infested with misinformation and manipulation. Beside the lying ads themselves, relying on a revenue stream entirely dependent on how many ads are seen severely affects the moral choices of those who decide what gets produced and how its presented. What are the costs of a misinformed and variously manipulated citizenry, of distortions to the free-market?
Knowledge and discourse are the lifeblood of both democracy, free markets, progress. The web, from the little scammy websites to the big brand ones that so many blindly trust, has a huge influence on who we vote for, what we buy, and most importantly, what we believe.
There is no free lunch, and there is no free web. This "free" ad-"supported" web we have is much too expensive.
[4] Lloyd made his pitch, proposing a quantum version of Google’s search engine whereby users could make queries and receive results without Google knowing which questions were asked. The men were intrigued. But after conferring with their business manager the next day, Brin and Page informed Lloyd that his scheme went against their business plan. "They want to know everything about everybody who uses their products and services," he joked. - Wired, http://www.wired.com/wiredscience/2013/10/computers-big-data...
I think I can see both sides. If all meaningful stats you are collecting are going down, I think something is definitely wrong. There are basically three options:
1. Fix/revert the update so that those stats increase again. This is probably the easiest/quickest option, especially the revert, but it obviously sucks to have "wasted" time on the updates.
2. Collect new/more stats that you deem meaningful and see if those have increased. This is the one that I like best from the perspective of a lowly programmer who worked hard on the updates, because it provides the possibility that even though existing stats have worsened, the updates are still empirically better. Of course, it's hard to figure out what new stats to look for, and whether or not to deem them truly meaningful. An oversimplified example would be if all you're looking at is ad conversions, maybe you should start looking at user retention as well, because even if ad conversions decrease, a large increase in retention can in the long (or even short) run lead to more revenue.
3. Ignore the numbers on the prediction that the numbers will increase over time. This one is also appealing, but will and probably should be rejected by management.
Something that also needs to be recognized is that "significantly more modern and easier to use" is a subjective claim, and is not necessarily "better" for whatever definition of "better" the company is using.
Nice theory, but I think Dustin's conclusions are wrong.
I got to use this alternative design on my second Facebook account that I used for app development, while my personal account didn't have it enabled. I really disliked like the new sidebar design. The concept was similar to what GMail has done lately, with text links replaced by only graphical icons. I found it really difficult to remember what each icon linked to, and I'd have to go through and hover over each icon one by one.
My theory (which I think has as much evidence to back it up as Dustin's) is that if the feed performed better in this design, it was because the poorly designed menu made it more difficult to navigate the rest of the site!
This is a terribly pernicious trend in all sorts of software, not just GMail. I really hope that the fad dies out soon, because it's preventing me form effectively using software that otherwise may have been great.
I wouldn't be surprised if a lot of people (including myself) just need the feed. I personally preferred the clean approach because outside of Events, I never need the left navigation. Now it obnoxiously takes up a segment of the page and the main content in the center feels tight especially with the bigger typography.
It wasn't stuck at just the icons, it let you expand to see words as well. It was much better in my opinion and I rarely had it at the collapsed state like his screenshot shows.
I already never leave my feed—and all they've done by streamlining it is make ads a full third of the screen. Ads for me, a 22 year old, to bulk up, grow my hair back, certain for any male over 55 to like.
This article makes one big unstated assumption: that users wanted the news feed to change. In fact, users didn't want the news feed to change. Users hate change. And when I say that I don't mean that users are stupid and hate good things. Users have good reasons for hating change that's forced on them: it reduces the value of their previous experience and requires extra time and effort on their part; effort that they'd rather be spending on things they actually care about.
Users didn't want the news feed to change, and the users were right.
There's a subtle distinction to be made here. The argument is that a better user experience was abandoned in favor of a worse user experience that generated more impressions per user.
It might be true that users would have an immediately negative reaction to change; I suppose it's impossible to know for sure. But that doesn't mean that the new design wouldn't have been better at providing the functionality that Facebook purports to offer.
Even if you're right, there's a similar long-term-vision vs. short-term-incentives situation. The overhead of having to "learn" a new system (which is already and would have remained fairly passive and simple to use) becomes less reasonable to hate when amortized over years of improved performance.
I don't really use Facebook much and am agnostic about the extent to which one site or another would better serve users. It's ambiguous to me what Facebook truly sees as its function / purpose anyway, so the criteria is murky here.
But the argument in this article isn't necessarily assuming anything about what users want, rather it purports to know from a design perspective what will ultimately serve them better.
Are the users' opinions even relevant to Facebook? Remember, the users aren't their "customers," they're their product. Their actual "customers" are the advertisers. If the new news feed is making their advertisers happy (and bringing revenue into Facebook), then that's what they optimize for.
Would be really interesting to put a number on "users hate change".
Based on my own experience at a company where we actually researched this stuff, the number I would forward is 30%. Given an existing user base, on average 30% will hate any given change to their user experience, independent of whether the that experience is actually worse or better.
> Users didn't want the news feed to change, and the users were right.
This time maybe (you'd have to look at the stats, not at the ones shouting the loudest). Usually they're not though. Users will complain about every single thing that changes, even if it objectively changes for better. Just look at the bad press Google is getting for... forcing users to use HTTPS.
Sometimes it's because benefits are not immediately apparent. Sometimes it's attention whoring (and ad-revenue for your site). And then sometimes it's just that complaining about how everything sucks is a popular way of doing smalltalk these days. But the truth is, one just really has to ignore what users "want". They'll come around anyway. Ford's faster horses, and the like.
There will always be users who have strong, nonstandard design preferences. For those we invented userscripts and bookmarklets.
I can say that I have disliked more or less every change Facebook has made since I joined. I, like a large section of early adopters, only keep my account around because of an attachment to old social connections.
That's why the team behind the Paper app were smart to make it a separate app instead of a redesign. They know the consistent negative user reaction when facebook redesigns.
You make me think of QWERTY keyboards and how they remain the most common format globally in spite of being intentionally designed to be inefficient.
Some people seem to believe in an "ideal world" and let that interfere with interacting with actual reality in a practical way. Things need to be backwards compatible with user experience to succeed. New designs sometimes simply are not that.
If you trust your metrics and nothing else, you have to be very sure that your metrics encompass every aspect of the reality you are modelling. If they just tell you about clicks and sales, they might be missing longer-term objectives like user satisfaction and retention.
I've never worked at Google and I can respect the sentiment, but it seems to me that if you are a skilled designer that Google's massive audience and experimental capabilities can cut both ways. It means that you have to prove even the most minor design changes, but it also means that you have the ability to really prove your work is helpful and not fool yourself.
If you make a few small wins you surely can build up the credibility to take bigger steps -- the catch is though in my experience bigger steps never improve metrics anyway and are a waste of time since you end up having to throw more work out. I would kill to be able to run a 0.1% test on a large change and not disrupt things too much, while still getting statistical significance. Few places other than Google seem to have the traffic to do that.
The core of Dustin's argument is that Facebook may not have been patient enough; they should have trusted in their beautiful new design and waited long enough for the benefits to bear fruit.
However, it's a cheap argument to make, because the Hard Thing is to decide how many months of crappy numbers are you going to withstand before you admit that your Beautiful New Design in fact isn't any good?
Six months? Two years?
And it's not just revenue you look at. How's overall engagement? Sharing rates? Communication? Discovery?
The article is a shallow snipe; the real issues here are hard, interesting, and unexplored by this piece.
The alternate design isn't "performing too well" by not telling you which of your friends are online to chat with, like the current version does. It's just decluttering, and relegating that important function to one of many miniscule, unlabelled icons. It's not "performing too well" by rendering links in the same colour as body text, and making the search function look like a header: it's just making them subtly less obvious, which matters when your users are in the hundreds of millions and some of them really aren't that savvy.
(Possibly it matters even more with casual users who are web-savvy, in that you're missing an opportunity to encourage them to search by prominently positioning the sort of medium white box that makes them think about searching)
Whatever is cleanest and most elegant is not necessarily the most user-friendly design, never mind the optimal design from the point of view of user engagement.
Very good observations. You made obvious to me most of what I did not like in that new design. My first reaction to the look of the new design is that is made each entry much larger filling the screen making the text less noticeable and having fewer posts on a page. A useful design for me will be concise so I can scan more quickly.
I'll give another way to look at it. Good design is by no means the same as optimal design. A lot of beautiful designs done by talented designers end up being worse than what was before. They might be prettier, but they are very likely worse by many conversion metrics.
Don't think of it in terms of pure design. Think of it in terms of cost. Everything has a cost and sometimes good design's real cost is in user behavior. Pageviews and time on site could go down because people aren't going through so many steps to get to what they want. There are a lot of metrics that aren't that useful without the context of the ultimate conversion numbers for your site/app/product/project.
Facebook and Google are advertising companies. The financial metric they care about is advertising revenue per user and number of users. It's not much different than a SAAS app in that way. Other metrics are important, but that is the metric that pays the bills.
A beautiful design that doesn't improve the core metrics is like a multi million dollar super bowl commercial that flops. Sure, it might be really cool and well produced, but if it doesn't sell your product, you might as well light that money on fire. The net effect is the same.
The underlying assumption being something like "If you're not increasing the amount of money you make, you are of no value"
Facebook says it's values are connecting people and other helpful ideas for a reason. If they created a design that helped people connect more fluidly, it'd still be valuable, even if it didn't increase their revenue. I find your comment a bit cynical and oversimplified.
Here's a hypothetical counterpoint to your proposition here:
A search company makes money from ads, which are unobtrusive text links which are clearly separated from search results.
The search company is forever trying to improve metrics, and in this quest, it pushes more intrusive monitoring of users, and the adverts become more and more like search results at the top of their results. According to their metrics, this is a winning move - more and more people are clicking ads, and advertisers love the stats on users.
The search company slowly starts to lose users, because they don't like the privacy issues and the intrusive ads in their search, even if more people click on them.
A few years later, the search company starts to lose its dominance of the field and hence all the money they made from more and more ads is now drying up, but every change they make to increase clicks on ads is actually having a worse effect on usage a few months down the line, even if it helps their metrics in the short term. No-one notices because the users have always been there, and this is not a primary metric or one that is easily correlated with changes.
Was this a good strategy? It was certainly a local maximum, and all the numbers went up, but was it in their long term interests?
I agree for someone like Google or FB the metric which is important is selling ads (that is their business after all), but both companies actually have two sets of users - one set are the users who read the ads, and one set are the advertisers - both are vitally important to their brand and continuing profit. Only one of those is represented in the advert CTR metric, and only one directly makes money, but both are vitally important to continuing profits.
Now you can change your metrics to try to measure user retention, but happiness and satisfaction are notoriously slippy concepts, and people often don't even know what they want before they see it and use it for a while (e.g. users of yahoo or one of the other portals would not have voted for changes to make it more like google), people hate any change, and haters are the most vocal. All those things completely skew any attempt to get numbers on customer satisfaction.
Of course, because money is the only criteria that is important to any business. There couldn't possibly be anything else, like a sense of mission or purpose, that could inform decisions.
Once you take outside money, you have to give up the sense of purpose if it conflicts with collecting money.
This is more an issue of optimization versus long term vision. Some firms can act like Apple, ignore feedback, and invent the future. That's very rare. The rest must react to feedback.
There is only one core product that Facebook has. It's not a platform for anything other than that. Any new features must pass that test, and the implementation of this one didn't.
What I take from this is that Dustin Curtis plays Farmville and is a member of a shadowy group named "secret group".
Actually, the question of the piece is a good one. It's really about what you're optimizing for. As every halfway decent manager knows, you get what you measure. Which means deciding what to measure is one of the most important decisions you can make.
So, in this case, do you measure user engagement time for individual sessions? Or is there some sort of "engagement longevity" which might show a better timeline keeps people visiting more often over a longer period of time?
The other possible approach would be to see what could be done to make events and profile pages more appealing to spend time on. There may not be a way to do that if the timeline satisfies people, but it would be worth investigating.
And there's things you can't measure, like long-term reputation and habit forming. I think long-term reputation is how a lot of good startups become successful against big, organized, optimizing companies.
The thing that makes me really insane about this approach is how mindless it ends up being. If you're going to abdicate all responsibility to some set of metrics, it's the opposite of thinking. The numbers become a capitalist lullaby that switches everybody's brains off.
If you're going to work strictly by the short-term numbers, you might as well be the bubonic plague. "Good news! We're up 32% in London! Quarterly bonuses for all the fleas, and gift cards for the rats at the all-hands!"
This is the result of placing the burden of proof on vision and innovation. Companies optimize for local maxima at the expense of global maxima because proving that both the mountain exists in the distance and that you can reach it, turns out to be extremely difficult.
And so, in companies like Facebook and Google, it doesn't matter what you know, it only matters what you can prove. Meanwhile your competitors in the market are unburdened by the need for proof and shout down at you from the mountain in the distance when they arrive.
I worked on the design of the desktop Facebook News Feed. Just posted a response to the article here: https://medium.com/p/ed75a0ee7641
Actually, the older version of the design we tested would have been positive for revenue had we shipped it. But there were a number of other issues that made it harder for people to use (which also resulted in them liking it less.)
A depressing thought but important especially if you are running a startup. It's ok for Facebook to take a hit like this and revert but if you spend 6 months at your startup redesigning your product and even though people like it your revenues suffer massively you might not even have time to test and revert back.
It seems Facebook really is going through the Google phases, and they've always wanted to "be Google" anyway. Right now they're in the Google phase of 5-7 years or so ago, when Google was still doing everything by the numbers, even at the expense of UI and UX.
Just like Google of 5-7 years ago, they're also spreading their focus on many projects, and in a few years probably forgetting about them and ignoring them, if they don't turn into big cash cows for them almost immediately. Then expect Facebook to kill a lot of services, just like Google did.
This is really an instance of the general phenomenon that the eminent tend to take fewer risks. A change that decreases revenue isn't necessarily bad; it may even yield a net profit in the long run. It's just perceived as a risk because things like user happiness and product culture (a) can't be as easily measured and (b) don't yield results for a while.
I think this is actually a rational-- or at least natural-- course of action. As you get more eminent, the stakes are also higher, and when you have more to lose you tend to take less risk. In fact, it'd be surprising if a big company continued taking risks by trusting non-structural decisions.
This is probably related to the phenomenon that large organizations tend to fall into bureaucracy. In fact the two questions are probably overlapping, if not identical. How can you grow big and famous and take on big responsibilities without losing your ability to trust your intuition and care about the feel and usability of the product? How can you stop yourself from degenerating into bureaucracy?
I'm pretty confident it's possible. Steve Jobs managed it. My own hunch is that the trick is to hire people who don't care about money too much. The kind of people who think, if we lose a bit of revenue, who cares? Which is paradoxical, my hunch continues, because people like this will eventually make better products in the long run, and end up increasing revenue in a thousand different little ways.
... or that beauty doesn't necessarily convert better. We've seen this time and time again with sites like Craigslist and Ebay and recently 42Floors wrote about a similar experience when experimenting with radically different search result treatments.
I really do like the new treatment and I think they should have gone with this and figured out how to recover the revenue stream later. Given how much Facebook traffic is going to mobile instead of desktop, this wouldn't have a large impact over the long run.
And your solution is to do LESS TESTING? We don't know what we're doing, so let's cut back on the amount of data we can use to inform our decisions?
> "We are slaves to the numbers. We don’t operate around innovation. We only optimize."
I don't see why numbers should ever stop you from innovating. The difference between "innovating" and "optimizing" is just a difference of scale. You can make a huge change to your layout or site function and look at the numbers it the same way you'd look at a font and color change.
The quote above seems to say that people shouldn't make decisions based on numbers, and that's absurd for a company like Facebook. What should be the basis of their decisions then? Management's gut reaction? Whoever feels the strongest about a change wins?
Customer surveys and user metrics matter - both are often numbers. The real issue here isn't that Facebook uses numbers too much. If they made the wrong choice, it's because they put too much emphasis on the wrong numbers.
The point is about design being held back or distorted by numbers - which are often a false prophesy when it comes to UX improvements.
Nobody is suggesting that numbers don't matter, but if you can't decide on a border thickness of 1 or 2 pixels without numbers to tell you which to go with, you've lost your mojo. And that matters in the long term.
When the aim is to increase length of time on the site or in the app, the numbers will "tell you" to make the interface more complicated, to hide the sign-out button (common practice), to hide the exit button (like Dropbox did on the desktop application), to force dependencies between unrelated services (Google does this a lot).
When the aim is to increase clicks on Ads, the numbers will tell you to make ads look like normal posts. The numbers will tell you to autoplay videos on tabloid news article pages. But all these "optimizations" are not unlike speed cameras placed at the bottom of the hill where lots of people can't help bumping over the speed limit for a couple of seconds. Good for revenue, bad vibes in every other way.
I think the problem is that we don't seem to have hypothesis' that guide our interpretation of the numbers.
In the absence of a guiding framework on how to interpret your data, you're bound to min-max individual criteria; we'll never know if we're just in a local maxima, however.
I'm not saying this is an easy problem, mind you. It's easy to wax poetic about having your culture guide decisions when you can point at a graph and say "yep, we're making $xx,000 less per hour".
I did some design and coding work for the adult industry for a while. The company I worked for had one large members area with all kinds of niches, with thousands of sites acting as doorways into it. Instead of a nice overview, and a smooth experience, they had built in tons of tricky ways to delay the user getting to the content. From loading delays to tricky dropdowns instead of simple buttons.
Users had access to the site as long as they stayed on the phone to our special 2 dollar per minute phone line.
It worked, but it was a pain to work for a company like that. I was fresh out of school and just wanted to get better at my trade, but wasn't allowed to do the best I could. Frustrating.
Needless to say, things have changed in that industry, gotten a lot trickier, and the company has had to switch into different avenues. They now offer payment solutions and run a huge dating site.
[+] [-] steven2012|12 years ago|reply
When you are stuck in a company that can't innovate because a shitty site leads to more money due to inertia, then you know you are on your way down. This leads to your best developers thinking "what the fuck did I waste all my time for?" and they will leave in no uncertain terms.
[+] [-] ignostic|12 years ago|reply
> "This is truly a nightmare scenario for any CEO: do you take the risk and proceed with the better user experience/product at the expense of short term numbers–with no promise that the better design will actually lead to long-term benefits–or do you scrap the new design and start over?"
Would the simplified, modern design have led Yahoo into a new era of profitability? Maybe. Or maybe they'd just have a nicer site that made less money for a year.
Our team made a site that made less per visitor, but we felt like it was a better-looking and more useful site. We decided to stick with it and work out the kinks, but ~6 months went by without reaching our old sites' benchmark. There was a ton of pressure on me to make it work or admit defeat and revert. I nearly gave in as the stress built, but eventually revenue surpassed the old site. I later learned that I was almost fired over it.
My story has a happy ending, but just barely. I have sympathy for the people making these decisions.
[+] [-] eevilspock|12 years ago|reply
IT'S NOT FREE
We’re not Facebook’s customers, advertisers are (more on this below). But we are the advertiser’s customers, and the cost of the "free lunch" is simply shifted to the price of the things we buy from them. In other words we still end up paying for the full cost of Facebook (and even more, as I'll get to next). Costs may even shift regressively, to advertised products predominately consumed by those with lower incomes, in which case the poor are subsidizing the better off.
IT'S MORE EXPENSIVE...
Not only are you still paying for the full cost of the Facebook product you use, you are paying for all the advertising overhead: the costs of its advertising technology and infrastructure (huge, btw), the agency and creative costs (Don Draper and company have to pay for the hookers and scotch somehow, not to mention what’s-his-name who basically just lounges in his office barefoot thinking Japanese), and the advertiser's big marketing departments (that often outnumber and outspend the people making the product!).
The best minds of my generation are thinking about how to make people click ads. That sucks. – Jeff Hammerbacher, fmr. Manager of Facebook Data Team, founder of Cloudera
So in addition to the original product cost and the ad overhead costs, you are also paying the opportunity cost of an inferior product (as Dennis Curtis points out in the OP) as well as the engineering costs of figuring out how to optimize ad revenue, because that’s what happens when websites have to design to please advertisers over pleasing us, the users. Dalton Caldwell makes this point comparing Sourceforge to Github[1]. As ergo says in a comment[2], “If the new news feed is making their advertisers happy (and bringing revenue into Facebook), then that's what they optimize for.” As jfoster says in a comment[3], “Ad-supported models untie the relationship between UX and revenue.”
Furthermore, our identities and privacy are bought and sold to the highest bidders. And where do the bidders get their money? From us of course! A double whammy! We're trading our privacy for free product? Bullshit. We get personalization? Bullshit. Personalization means optimizing something for me, not optimizing for the advertiser. Again, we're not the real customer. We’re certainly not Google’s[4][5].
WAIT. IT'S EVEN WORSE...
Advertising has us chasing cars and clothes, working jobs we hate so we can buy shit we don’t need. – Tyler Durden, Fight Club
Think of the social costs of advertising. The web is infested with misinformation and manipulation. Beside the lying ads themselves, relying on a revenue stream entirely dependent on how many ads are seen severely affects the moral choices of those who decide what gets produced and how its presented. What are the costs of a misinformed and variously manipulated citizenry, of distortions to the free-market?
Knowledge and discourse are the lifeblood of both democracy, free markets, progress. The web, from the little scammy websites to the big brand ones that so many blindly trust, has a huge influence on who we vote for, what we buy, and most importantly, what we believe.
There is no free lunch, and there is no free web. This "free" ad-"supported" web we have is much too expensive.
—
[1] http://daltoncaldwell.com/an-audacious-proposal
[2] https://news.ycombinator.com/item?id=7484075
[3] https://news.ycombinator.com/item?id=7484442
[4] Lloyd made his pitch, proposing a quantum version of Google’s search engine whereby users could make queries and receive results without Google knowing which questions were asked. The men were intrigued. But after conferring with their business manager the next day, Brin and Page informed Lloyd that his scheme went against their business plan. "They want to know everything about everybody who uses their products and services," he joked. - Wired, http://www.wired.com/wiredscience/2013/10/computers-big-data...
[5] Google's once famous clean and neutral search results are now cluttered and biased (http://arstechnica.com/business/2013/10/new-banner-ads-push-...).
[+] [-] baddox|12 years ago|reply
1. Fix/revert the update so that those stats increase again. This is probably the easiest/quickest option, especially the revert, but it obviously sucks to have "wasted" time on the updates.
2. Collect new/more stats that you deem meaningful and see if those have increased. This is the one that I like best from the perspective of a lowly programmer who worked hard on the updates, because it provides the possibility that even though existing stats have worsened, the updates are still empirically better. Of course, it's hard to figure out what new stats to look for, and whether or not to deem them truly meaningful. An oversimplified example would be if all you're looking at is ad conversions, maybe you should start looking at user retention as well, because even if ad conversions decrease, a large increase in retention can in the long (or even short) run lead to more revenue.
3. Ignore the numbers on the prediction that the numbers will increase over time. This one is also appealing, but will and probably should be rejected by management.
Something that also needs to be recognized is that "significantly more modern and easier to use" is a subjective claim, and is not necessarily "better" for whatever definition of "better" the company is using.
[+] [-] akamaka|12 years ago|reply
I got to use this alternative design on my second Facebook account that I used for app development, while my personal account didn't have it enabled. I really disliked like the new sidebar design. The concept was similar to what GMail has done lately, with text links replaced by only graphical icons. I found it really difficult to remember what each icon linked to, and I'd have to go through and hover over each icon one by one.
My theory (which I think has as much evidence to back it up as Dustin's) is that if the feed performed better in this design, it was because the poorly designed menu made it more difficult to navigate the rest of the site!
[+] [-] epistasis|12 years ago|reply
This is a terribly pernicious trend in all sorts of software, not just GMail. I really hope that the fad dies out soon, because it's preventing me form effectively using software that otherwise may have been great.
[+] [-] colmvp|12 years ago|reply
[+] [-] selectout|12 years ago|reply
[+] [-] duaneb|12 years ago|reply
[+] [-] modeless|12 years ago|reply
Users didn't want the news feed to change, and the users were right.
[+] [-] ForrestN|12 years ago|reply
It might be true that users would have an immediately negative reaction to change; I suppose it's impossible to know for sure. But that doesn't mean that the new design wouldn't have been better at providing the functionality that Facebook purports to offer.
Even if you're right, there's a similar long-term-vision vs. short-term-incentives situation. The overhead of having to "learn" a new system (which is already and would have remained fairly passive and simple to use) becomes less reasonable to hate when amortized over years of improved performance.
I don't really use Facebook much and am agnostic about the extent to which one site or another would better serve users. It's ambiguous to me what Facebook truly sees as its function / purpose anyway, so the criteria is murky here.
But the argument in this article isn't necessarily assuming anything about what users want, rather it purports to know from a design perspective what will ultimately serve them better.
[+] [-] erbo|12 years ago|reply
Harsh, I know, but that's the reality.
[+] [-] harryf|12 years ago|reply
Based on my own experience at a company where we actually researched this stuff, the number I would forward is 30%. Given an existing user base, on average 30% will hate any given change to their user experience, independent of whether the that experience is actually worse or better.
[+] [-] TeMPOraL|12 years ago|reply
This time maybe (you'd have to look at the stats, not at the ones shouting the loudest). Usually they're not though. Users will complain about every single thing that changes, even if it objectively changes for better. Just look at the bad press Google is getting for... forcing users to use HTTPS.
Sometimes it's because benefits are not immediately apparent. Sometimes it's attention whoring (and ad-revenue for your site). And then sometimes it's just that complaining about how everything sucks is a popular way of doing smalltalk these days. But the truth is, one just really has to ignore what users "want". They'll come around anyway. Ford's faster horses, and the like.
There will always be users who have strong, nonstandard design preferences. For those we invented userscripts and bookmarklets.
[+] [-] colechristensen|12 years ago|reply
[+] [-] ilovecomputers|12 years ago|reply
[+] [-] anon808|12 years ago|reply
[+] [-] Mz|12 years ago|reply
Some people seem to believe in an "ideal world" and let that interfere with interacting with actual reality in a practical way. Things need to be backwards compatible with user experience to succeed. New designs sometimes simply are not that.
[+] [-] grey-area|12 years ago|reply
http://stopdesign.com/archive/2009/03/20/goodbye-google.html
If you trust your metrics and nothing else, you have to be very sure that your metrics encompass every aspect of the reality you are modelling. If they just tell you about clicks and sales, they might be missing longer-term objectives like user satisfaction and retention.
[+] [-] gfodor|12 years ago|reply
If you make a few small wins you surely can build up the credibility to take bigger steps -- the catch is though in my experience bigger steps never improve metrics anyway and are a waste of time since you end up having to throw more work out. I would kill to be able to run a 0.1% test on a large change and not disrupt things too much, while still getting statistical significance. Few places other than Google seem to have the traffic to do that.
[+] [-] harryf|12 years ago|reply
[+] [-] dsjoerg|12 years ago|reply
However, it's a cheap argument to make, because the Hard Thing is to decide how many months of crappy numbers are you going to withstand before you admit that your Beautiful New Design in fact isn't any good?
Six months? Two years?
And it's not just revenue you look at. How's overall engagement? Sharing rates? Communication? Discovery?
The article is a shallow snipe; the real issues here are hard, interesting, and unexplored by this piece.
[+] [-] notahacker|12 years ago|reply
Whatever is cleanest and most elegant is not necessarily the most user-friendly design, never mind the optimal design from the point of view of user engagement.
[+] [-] jbaskette|12 years ago|reply
[+] [-] programminggeek|12 years ago|reply
Don't think of it in terms of pure design. Think of it in terms of cost. Everything has a cost and sometimes good design's real cost is in user behavior. Pageviews and time on site could go down because people aren't going through so many steps to get to what they want. There are a lot of metrics that aren't that useful without the context of the ultimate conversion numbers for your site/app/product/project.
Facebook and Google are advertising companies. The financial metric they care about is advertising revenue per user and number of users. It's not much different than a SAAS app in that way. Other metrics are important, but that is the metric that pays the bills.
A beautiful design that doesn't improve the core metrics is like a multi million dollar super bowl commercial that flops. Sure, it might be really cool and well produced, but if it doesn't sell your product, you might as well light that money on fire. The net effect is the same.
[+] [-] droopyEyelids|12 years ago|reply
Facebook says it's values are connecting people and other helpful ideas for a reason. If they created a design that helped people connect more fluidly, it'd still be valuable, even if it didn't increase their revenue. I find your comment a bit cynical and oversimplified.
[+] [-] grey-area|12 years ago|reply
A search company makes money from ads, which are unobtrusive text links which are clearly separated from search results.
The search company is forever trying to improve metrics, and in this quest, it pushes more intrusive monitoring of users, and the adverts become more and more like search results at the top of their results. According to their metrics, this is a winning move - more and more people are clicking ads, and advertisers love the stats on users.
The search company slowly starts to lose users, because they don't like the privacy issues and the intrusive ads in their search, even if more people click on them.
A few years later, the search company starts to lose its dominance of the field and hence all the money they made from more and more ads is now drying up, but every change they make to increase clicks on ads is actually having a worse effect on usage a few months down the line, even if it helps their metrics in the short term. No-one notices because the users have always been there, and this is not a primary metric or one that is easily correlated with changes.
Was this a good strategy? It was certainly a local maximum, and all the numbers went up, but was it in their long term interests?
I agree for someone like Google or FB the metric which is important is selling ads (that is their business after all), but both companies actually have two sets of users - one set are the users who read the ads, and one set are the advertisers - both are vitally important to their brand and continuing profit. Only one of those is represented in the advert CTR metric, and only one directly makes money, but both are vitally important to continuing profits.
Now you can change your metrics to try to measure user retention, but happiness and satisfaction are notoriously slippy concepts, and people often don't even know what they want before they see it and use it for a while (e.g. users of yahoo or one of the other portals would not have voted for changes to make it more like google), people hate any change, and haters are the most vocal. All those things completely skew any attempt to get numbers on customer satisfaction.
[+] [-] swombat|12 years ago|reply
[+] [-] res0nat0r|12 years ago|reply
[+] [-] mathattack|12 years ago|reply
This is more an issue of optimization versus long term vision. Some firms can act like Apple, ignore feedback, and invent the future. That's very rare. The rest must react to feedback.
[+] [-] kordless|12 years ago|reply
[+] [-] wrongc0ntinent|12 years ago|reply
[+] [-] waterlesscloud|12 years ago|reply
Actually, the question of the piece is a good one. It's really about what you're optimizing for. As every halfway decent manager knows, you get what you measure. Which means deciding what to measure is one of the most important decisions you can make.
So, in this case, do you measure user engagement time for individual sessions? Or is there some sort of "engagement longevity" which might show a better timeline keeps people visiting more often over a longer period of time?
The other possible approach would be to see what could be done to make events and profile pages more appealing to spend time on. There may not be a way to do that if the timeline satisfies people, but it would be worth investigating.
[+] [-] ploxiln|12 years ago|reply
[+] [-] pavel_lishin|12 years ago|reply
Sounds like we should be measuring our decisions about what to measure!
[+] [-] wpietri|12 years ago|reply
If you're going to work strictly by the short-term numbers, you might as well be the bubonic plague. "Good news! We're up 32% in London! Quarterly bonuses for all the fleas, and gift cards for the rats at the all-hands!"
[+] [-] eldude|12 years ago|reply
And so, in companies like Facebook and Google, it doesn't matter what you know, it only matters what you can prove. Meanwhile your competitors in the market are unburdened by the need for proof and shout down at you from the mountain in the distance when they arrive.
[+] [-] bqe|12 years ago|reply
How do you encourage finding a global maxima within your company?
[+] [-] joulee|12 years ago|reply
Actually, the older version of the design we tested would have been positive for revenue had we shipped it. But there were a number of other issues that made it harder for people to use (which also resulted in them liking it less.)
[+] [-] nedwin|12 years ago|reply
Don't seem to hear much from the inside since the funding announcement over 12 months ago...
[+] [-] k-mcgrady|12 years ago|reply
[+] [-] higherpurpose|12 years ago|reply
Just like Google of 5-7 years ago, they're also spreading their focus on many projects, and in a few years probably forgetting about them and ignoring them, if they don't turn into big cash cows for them almost immediately. Then expect Facebook to kill a lot of services, just like Google did.
[+] [-] coppolaemilio|12 years ago|reply
[+] [-] _xhok|12 years ago|reply
I think this is actually a rational-- or at least natural-- course of action. As you get more eminent, the stakes are also higher, and when you have more to lose you tend to take less risk. In fact, it'd be surprising if a big company continued taking risks by trusting non-structural decisions.
This is probably related to the phenomenon that large organizations tend to fall into bureaucracy. In fact the two questions are probably overlapping, if not identical. How can you grow big and famous and take on big responsibilities without losing your ability to trust your intuition and care about the feel and usability of the product? How can you stop yourself from degenerating into bureaucracy?
I'm pretty confident it's possible. Steve Jobs managed it. My own hunch is that the trick is to hire people who don't care about money too much. The kind of people who think, if we lose a bit of revenue, who cares? Which is paradoxical, my hunch continues, because people like this will eventually make better products in the long run, and end up increasing revenue in a thousand different little ways.
[+] [-] calbear81|12 years ago|reply
I really do like the new treatment and I think they should have gone with this and figured out how to recover the revenue stream later. Given how much Facebook traffic is going to mobile instead of desktop, this wouldn't have a large impact over the long run.
[+] [-] ignostic|12 years ago|reply
And your solution is to do LESS TESTING? We don't know what we're doing, so let's cut back on the amount of data we can use to inform our decisions?
> "We are slaves to the numbers. We don’t operate around innovation. We only optimize."
I don't see why numbers should ever stop you from innovating. The difference between "innovating" and "optimizing" is just a difference of scale. You can make a huge change to your layout or site function and look at the numbers it the same way you'd look at a font and color change.
The quote above seems to say that people shouldn't make decisions based on numbers, and that's absurd for a company like Facebook. What should be the basis of their decisions then? Management's gut reaction? Whoever feels the strongest about a change wins?
Customer surveys and user metrics matter - both are often numbers. The real issue here isn't that Facebook uses numbers too much. If they made the wrong choice, it's because they put too much emphasis on the wrong numbers.
[+] [-] exodust|12 years ago|reply
Nobody is suggesting that numbers don't matter, but if you can't decide on a border thickness of 1 or 2 pixels without numbers to tell you which to go with, you've lost your mojo. And that matters in the long term.
When the aim is to increase length of time on the site or in the app, the numbers will "tell you" to make the interface more complicated, to hide the sign-out button (common practice), to hide the exit button (like Dropbox did on the desktop application), to force dependencies between unrelated services (Google does this a lot).
When the aim is to increase clicks on Ads, the numbers will tell you to make ads look like normal posts. The numbers will tell you to autoplay videos on tabloid news article pages. But all these "optimizations" are not unlike speed cameras placed at the bottom of the hill where lots of people can't help bumping over the speed limit for a couple of seconds. Good for revenue, bad vibes in every other way.
[+] [-] phillmv|12 years ago|reply
In the absence of a guiding framework on how to interpret your data, you're bound to min-max individual criteria; we'll never know if we're just in a local maxima, however.
I'm not saying this is an easy problem, mind you. It's easy to wax poetic about having your culture guide decisions when you can point at a graph and say "yep, we're making $xx,000 less per hour".
[+] [-] pjaspers|12 years ago|reply
[0] https://medium.com/launching-ux-launchpad/385ff833f9c8
[+] [-] richforrester|12 years ago|reply
Users had access to the site as long as they stayed on the phone to our special 2 dollar per minute phone line.
It worked, but it was a pain to work for a company like that. I was fresh out of school and just wanted to get better at my trade, but wasn't allowed to do the best I could. Frustrating.
Needless to say, things have changed in that industry, gotten a lot trickier, and the company has had to switch into different avenues. They now offer payment solutions and run a huge dating site.