When I was still on OKCupid, I once wrote a Chrome extension to hide the pictures. I found that I had, in general, a much better experience with the site - I'd actually read people's profiles, I sent better messages, and I got more responses. I eventually gave up on it when a site redesign changed some of the #ids I was depending on and I didn't feel like revamping the code. But interestingly, my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site. Setting up our first date, she was like "We don't need to exchange numbers, you know what I look like", and I was like "Actually, I'm not sure I do, here's mine." (Okay, I didn't actually tell her her pictures were terrible until we'd been dating for six months or so, but that was the general sentiment.)
I think it's one more example of when people's emotions and desires lead them to suboptimal outcomes. Most of the cues we associate with beauty and sexual desire evolved back in the savanna when health and fertility were very real risks; being able to pick up on which potential mates would be able to carry healthy offspring to term and nurture them until adulthood was very important then. Nowadays, the far greater risk is that you'll hate each other and fight all the time, but this has only been a concern since people started living long enough and in close enough proximity to care.
> my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site.
This is not what people mean when they say someone looks terrible in their pictures.
The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
Heh heh. I recently got a message from okcupid saying "Because of a diagnostic test, your match percentage with <user> was misstated as 31%. It is actually 91%. We wanted to let you know!" I had a feeling something like this might be involved. (I messaged that person anyway, because they had a funny username, so I guess I supported their point.)
"For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network."
I immediately dropped everything to read this blog post. I have been waiting for 3 years!
I am fascinated with the data and science behind human behaviour, and their blog posts were (now are) always really interesting.
Interesting that the author has a plug at the bottom for his new book, which he says he's been working on for the past 3 years. I guess his book writing got in the way of blogging.
I thought Experiment 2 was a great example of the pain of trying to fit your users' feelings into an "ideal" model.
When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing.
But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept.
Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively.
So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly.
So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it.
PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.
This phenomenon is actually quite common. In retail, it's called the customer funnel: http://en.wikipedia.org/wiki/Purchase_funnel. The idea is you want to capture as much awareness with something really simple, usually it's a brand name, e.g. Apple. When customers express more interest, by coming in Apple store/website, they are presented with more design/technical details. Finally, only when they show purchase intent, they are shown price/configuration/delivery options.
On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error.
For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore.
[1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them.
OKC experimented with the matching process -- the express purpose of their site. Facebook experimented with mood manipulation -- something they have no permission to do -- regardless of the fine print. Facebook is a very bad landlord, but people don't want to move.
But Facebook already displays posts based on an algorithm; it doesn't do it chronologically, and it's actually almost impossible for me to find a specific post in my timeline. Especially from the mobile site, it's very common that I see something in the moment between opening the app and my timeline updating, and never being able to see the update again.
Facebook is already making the decision as to whether to show you a post or not, and they make that decision based on what they think you will want to see and what you will engage with, all of which is fuzzy and subjective. They already manipulate things you see to encourage engagement; for example, if their algorithm shows that your engagement is dropping and you're about to leave, they'll show other people your profile and say 'do you know this person?', because someone adding you as a friend boosts your engagement.
So if you think that showing you posts which are happy or sad is manipulative, be aware that Facebook is already filtering your potential-friends list and showing you people solely to boost engagement (either yours or theirs). Looked at another way, this means that it's possible their algorithm isn't currently showing you people you might know because it's not as beneficial for them to make that connection for you yet.
So the purpose of Facebook's site is to get people to interact and generate behaviour, and now they're experimenting with that; which posts do we show? which do we hide? They haven't shown them all for ages, so this is just a tweak to their algorithm that they were testing.
What's really interesting here is how this could actually be used for good. Someone feeling crappy? Show them fewer negative posts and more positive posts. Maybe that will help. Show negative posts less often, and make society in general a little more positive.
Facebook didn't experiment with "mood manipulation" any more than OKC. Facebook experimented with changing the site layout and which content and how much to show. Exactly the same stuff as all sites test.
Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation.
But OKC's changes also affected people's moods. More than Facebook's, I would guess.
All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.
They experimented through misrepresentation. It's the latter that's the problem, not the former.
If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
I feel very "bearish" towards dating sites. People convince themselves they are saving time meeting more possible "matches" in less time but they forget that meeting online is not the same as in person, "turn-based" chat is different than "real-time" chat.
Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again.
edit:
Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.
For the record not every site does these type of experiments. I've been programming websites since the 90s and I've never experimented on my users like this. The most I've done is swap in and out user interface features to see which one has a better response in a typical A/B test. It's not the same as telling a lie to a user no matter how you split hairs about it. They're overstating their case because they happen to be in the business of matching people up.
Is anyone surprised that only 1/5 people who are 90% matches have a "conversation"? I don't use dating sites, but I'd think that people who do want to converse, and who better to converse with than 90% matches?
But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me.
Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.
If you want to understand this better I'd recommend going back through that Blog's archives, they talk extensively about this (TL;DR: Women are the minority on the site, and only respond to a small subset of messages, plus matches are personalty not looks and as this article indicates that is only 10% of people's opinions).
I don't think the sticky point is whether or not we should be experimenting with people to find what works and what doesn't. It's about respecting users as humans and not simply data points. How does OkCupid know that conversations "went deeper" or how often emails and numbers were exchanged? Were the participants aware of that level of scrutiny or is that tucked away in the ToS? Most people I know aren't even comfortable talking about their OkCupid, they use it as a means to an end and place trust in the service to be discrete. Do these experiments fall in line with user expectations?
I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.
For the emails and numbers, they can probably search for those in the messages quite effectively without many false-positives.
I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
>But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
Hm, that is unexpected. Wonder how long it will last.
> I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.)
It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.
I think the premise you are assuming is wrong: You assume that it is sending false data to them to say "this person (which our algorithm has deemed is a bad match), is a good match". The problem is that until the experiment is done, you don't how good a match that person actually is. So you aren't sending them "false data," because you don't know that it is "false data".
[+] [-] nostrademons|11 years ago|reply
I think it's one more example of when people's emotions and desires lead them to suboptimal outcomes. Most of the cues we associate with beauty and sexual desire evolved back in the savanna when health and fertility were very real risks; being able to pick up on which potential mates would be able to carry healthy offspring to term and nurture them until adulthood was very important then. Nowadays, the far greater risk is that you'll hate each other and fight all the time, but this has only been a concern since people started living long enough and in close enough proximity to care.
[+] [-] jessriedel|11 years ago|reply
This is not what people mean when they say someone looks terrible in their pictures.
The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
[+] [-] blt|11 years ago|reply
[+] [-] ssl_love|11 years ago|reply
"For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network."
https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-...
The Heartbreaking Truth About Online Dating Privacy: https://www.eff.org/press/releases/heartbreaking-truth-about...
[+] [-] 3pt14159|11 years ago|reply
[+] [-] rockdiesel|11 years ago|reply
They haven't updated their blog since April 2011.
[+] [-] ShaneOG|11 years ago|reply
[+] [-] cschmidt|11 years ago|reply
[+] [-] adamzerner|11 years ago|reply
[+] [-] zabcik|11 years ago|reply
[+] [-] discardorama|11 years ago|reply
I think that's around the time they were bought out by Match.com .
[+] [-] zach|11 years ago|reply
When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing.
But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept.
Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively.
So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly.
So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it.
PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.
[+] [-] boomzilla|11 years ago|reply
[+] [-] andrey-p|11 years ago|reply
On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error.
For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore.
[1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them.
Edit: tweaked phrasing as per comments.
[+] [-] Chinjut|11 years ago|reply
[+] [-] delecti|11 years ago|reply
[+] [-] interstitial|11 years ago|reply
[+] [-] danudey|11 years ago|reply
Facebook is already making the decision as to whether to show you a post or not, and they make that decision based on what they think you will want to see and what you will engage with, all of which is fuzzy and subjective. They already manipulate things you see to encourage engagement; for example, if their algorithm shows that your engagement is dropping and you're about to leave, they'll show other people your profile and say 'do you know this person?', because someone adding you as a friend boosts your engagement.
So if you think that showing you posts which are happy or sad is manipulative, be aware that Facebook is already filtering your potential-friends list and showing you people solely to boost engagement (either yours or theirs). Looked at another way, this means that it's possible their algorithm isn't currently showing you people you might know because it's not as beneficial for them to make that connection for you yet.
So the purpose of Facebook's site is to get people to interact and generate behaviour, and now they're experimenting with that; which posts do we show? which do we hide? They haven't shown them all for ages, so this is just a tweak to their algorithm that they were testing.
What's really interesting here is how this could actually be used for good. Someone feeling crappy? Show them fewer negative posts and more positive posts. Maybe that will help. Show negative posts less often, and make society in general a little more positive.
[+] [-] azakai|11 years ago|reply
Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation.
But OKC's changes also affected people's moods. More than Facebook's, I would guess.
All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.
[+] [-] yarrel|11 years ago|reply
If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
[+] [-] galfarragem|11 years ago|reply
Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again.
edit: Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.
[+] [-] ianstallings|11 years ago|reply
[+] [-] adamzerner|11 years ago|reply
But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me.
Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.
[+] [-] egypturnash|11 years ago|reply
One-liners and no picture are instant 'don't bother replying', and most people with more details tend to get weeded out for other reasons.
[+] [-] javert|11 years ago|reply
[+] [-] Someone1234|11 years ago|reply
[+] [-] cm2012|11 years ago|reply
[+] [-] mkr-hn|11 years ago|reply
[+] [-] rabbyte|11 years ago|reply
I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.
[+] [-] herge|11 years ago|reply
I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
[+] [-] tomjen3|11 years ago|reply
Hm, that is unexpected. Wonder how long it will last.
[+] [-] sliverstorm|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] jessriedel|11 years ago|reply
I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.)
It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.
[+] [-] lifeisstillgood|11 years ago|reply
Just a quote that resonates ... The design and architecture of our online environments will affect our happiness as much as anything else.
[+] [-] pervycreeper|11 years ago|reply
[+] [-] spott|11 years ago|reply
[+] [-] wcdolphin|11 years ago|reply
That to me is a distinct difference, both in practicality and morality.
[+] [-] tragomaskhalos|11 years ago|reply
[+] [-] bshoemaker|11 years ago|reply
[+] [-] krallja|11 years ago|reply