I'm the engineer he's referring to. I still get a laugh seeing Justin's amazement when we told him her name. To answer a few questions...
- This isn't the actual photo but looks similar, except that she was alone. Justin was afraid of reposting the same photo after the results of handing over a photo last time :)
- There was no useful exif data in the image.
- We knew her high school and approximate age in Austin, but were unable to find any yearbooks / class rosters online. (Idea not pursued: try to acquire a yearbook offline).
- Facebook Graph Search and LinkedIn with the criteria we knew didn't help.
- Google Image search didn't help.
> "I'm sure there's a lot of information missing from this story"
Quite true.
Ultimately we had several methods going at once to try to figure it out.
The one that came back with results first relies on some information Justin still doesn't know we had (though he would still be impressed), but other approaches we also expected to work would be possible with only what's in the story, and haven't yet been mentioned here.
* Everyone was at the original bar at the same time - someone else had photos from the night that had her and other people in them, and those people (her friends) could be found online.
* The bar had some kind of "sign-up for x" list, or some kind of contest, or some kind of photos of the night, which were helpful.
* You searched twitter/facebook/whatever for a girl posting about a boy she just met at bar X, blah blah
* We know she was going to some kind of event - you guys figured out what that was, and found a guest list or something.
* You guys did a some wi-fi sniffing and read a few emails. He did say he emailed you the photo, so you know which account to listen for.
You looked on Instagram and other for pictures taken at the bar, and you found one either taken by the woman in question or one of her friends with the mystery woman tagged.
Or maybe you knew that she wanted him to meet her at an event and you found a roster for that event- or ie if it was a real estate convention you would have a pretty good idea that she was a realtor and could look at websites for realtors in Austin.
Or maybe you checked his recent facebook friends, looked at his recent twitter followers, and everyone who liked/commented on his recent photos/posts on Instagram and whatever other social networks he's on- as well as the photos/posts he liked/commented on.
I think you took a (relatively) low-tech approach to this.
It was this line -
"On Thursday night, after exchanging texts all day, around 8:00pm local time the girl sent me a picture of her in order to convince me to meet her at an event she was going to."
Find the event Thursday night at 8:00pm in Austin and start tracking down from there.
post pic on craiglist/similar in Austin area. Say you found wallet/purse/camera/etc with her picture. Anyone that knows her please call/email w/ Name/Address for you to send back. Use highschool for validation when contacted. Might have some error.
The second photo you realized you had showed the subject holding the device used to take the picture sent to your colleague. You investigated that particular device and found a way to uniquely identify it's pictures based off some characteristic of it's photos, not necessarily EXIF, then scoured the Internet for photos with the same unique characteristic until you found one that revealed more information about the subject?
The team(including Justin: using first name here for convinience's sake) had a trip to Austin. Where they were should be a public knowledge. Perhaps they(you) started the search from there. A facebook search or so could tell some plans of people who would have fun at that particular place.
It is also possible that during the few days' correspondence one of the engineer guys had some other piece of info that we are not familiar with. Seeing how the engineers already knew her high school, I'm sure Justin talked about her with co-workers.
I think the key information that's missing here is what the engineers knew but Justin didn't know that they knew.(Sounds like a quote out of a Friends episode, mmm)
So you know the school and you have her picture. By her picture you can make a guess about her age. Then you search the web for graduates for particular dates from this particular school. It doesn’t have to be a yearbook or a class roster, people on LinkedIn mention the schools they’ve attended. Sooner or later you’ll find someone and then you can narrow the search. Even simpler, once you find someone you simply contact him with a fictional story as to why you’re searching this woman. From there on it just takes some old good fashioned social engineering.
You obviously knew something about her that made it possible, which you even admit Justin doesn't know you had. People are getting bored with your shenanigans, spill it!
Wherever you were that night had a Facebook page with an event created for the evening you were there. You uploaded your second picture to the Facebook event, or tagged the Facebook page of the bar you were at. Then someone tagged the girl in the photo, revealing her identity.
Justin's own phone leaked his location (through some google-lattitude, or Yelp like service or perhaps gps directly) while Justin drove (or rode with her in a taxi) to her home.
At this point he could not see my screen (he was standing directly in
front of me), but offered me a simple bet – He said if I sent him the
picture of the girl he could find out her name.
Reminds me of:
Sky Masterson: One of these days in your travels, a guy is going to show you a
brand-new deck of cards on which the seal is not yet broken. Then this guy
is going to offer to bet you that he can make the jack of spades jump out of
this brand-new deck of cards and squirt cider in your ear. But, son, do not
accept this bet, because as sure as you stand there, you're going to wind
up with an ear full of cider.
> Simplest explanation: Someone saw you and told him.
Of course one of the first things I did was ask another friend who I thought met her. But no, ultimately the answer did come from the internet, not from anyone I knew.
- This was just a fun bet between friends. Justin really didn't think it could be done and I wanted to show him how a little information goes a long way online.
- Justin treated this girl well and she even knew about this bet.
- "Look up online" != Stalking
- None of the discovered information has been posted publicly online. This isn't even her photo.
Yeah, this was my thought too. This whole situation is just creepy and its even creepier that so many people have a million other ways they would stalk somebody. Sure you can do all of these things, but if you have ethics you just don't.
I feel like the majority of folk are treating this as a purely intellectual challenge. Which it sort of is (to the HN crowd, since the actual photo information is missing) but please, let's not forget the context here.
If you know the high school and year(s), you go thru the friends list of every person on Facebook who did post their high school from that year (or nearby years), and does show their friends, until you see her face.
That's only a couple hundred kids, each with a couple hundred friends.
If you still get nothing, start doing friend requests for the people that don't show their friends. I'll bet you'd get another 25-30 kids that way.
I don't see why you'd need facial recognition software to go thru 20,000 pictures.
3/4ths of them would be the wrong gender/age and be immediately disqualified.
But if you're a programmer, you can probably scrape and aggregate to speed things up - drop out the males, other high schools, only show pictures once, etc.
I think what makes this feel creepy instead of cool is that you've just described an exploit without giving any thought to mitigating the threat.
So you've found an attack vector where you can get a woman's name and address off the 'net with a picture and a small amount of information. Maybe it doesn't work all the time, but it worked at least once. What are you going to do with that attack vector? Are you going to make an iPhone app, so that others can snap pictures of random women and recreate your exploit automatically? Or are you going to come up with ways the attack vector can be shut down, things that either individuals (eg, the woman herself) or organizations (like Facebook or Google) can do to block the attack?
One of the things that looks like a clue to me is that this the lead engineer of close.io. Browsing through their site, it looks like they do "telecom in an API"; it reminds me of Twilio and OneBox.
So, here's a thought. Let's say the Engineer has access to the SMS logs (NSA-style) for Justin's phone. (Maybe his phone goes through their system. But even if not, maybe it's company-provided, so there's an admin interface provided by the telecom that lists this information.)
Justin probably doesn't regularly text people in Texas, since the team is based out of Palo Alto. So look through his logs for a number in a Texas area code (http://en.wikipedia.org/wiki/List_of_Texas_area_codes) that he's been texting more often than usual.
That gets you the girl's phone number; probably a cell phone. Now, you need to go from the cell phone number to a last name. Reverse Phone Lookup probably won't work for this, although it's certainly worth a first try. More likely candidates: looking through the users table at close.io for a matching phone number, or doing a google search for the phone number. Or, try calling in the middle of the night and seeing what the voicemail says. Any of these approaches might work.
Once you've got a last name, grab a copy of the white pages. Most public schools publish their district boundaries, so go grab a copy of that. Look through all entries that match the last name, and see how many of them fall within (or close to, in the case politics made the boundary change) the district boundary. You probably will get a handful (maybe three or four). If you only get one, bingo!
If you get more than one, I'd call the cell phone to see if I can get a first name somehow. Then, I'd go through the landlines in the White Pages and call them one at a time, "Hello, may I speak to X? Sorry, wrong number." until one matches.
As someone who happens to be good at finding people on the internet (true confessions), the approach I would have taken would have been related to FB likes, or log-ins at the bar that they met on Facebook. I.e. they were all at the same bar in Austin, so search for people who have checked-in at the bar. And perhaps the other photo the engineers had was another location that she liked, or another piece of information about the bar in question. Then, it's just a matter of narrowing down results based on age and location. The fact that they knew the high school would have made this quest even easier. I once found an individual from a first name and the name of the bar they work at in (comically) downtown Austin. It took forever, but I had patience. :) The internet has all sorts of shortcuts to finding people if you know how to look!
If they knew her high school and were doing lots of batch processing, I wonder if they found digital copies of the yearbook and tried facial recognition. For a single subject, it doesn't even need to be particularly accurate- they can easily sift through twenty possible matches with their own eyes.
She looks pretty; At least for guys pretty = memorable. When I was single I was pretty bold with the ladies and I'd have flirted with her first (perhaps getting shot down, oh well), remembered at least some demographic information, and then used it to completely F with my coworker's mind for awhile. When I was younger I was quite the practical joker.
Note the convoluted written responses about not talking to anyone he knew and so forth which technically does not exclude the girl, herself, saying her own name.
The other alternative is she's somebody's buddy/family member so it would be trivial to know before hand that she's so and so's ex or sister or whatever and given that intel, figure it out.
I assume the author is not a Facebook friend with the girl. Otherwise it would have been too easy.
But friends of friends is not the only privacy leak on Facebook. There is a very old feature (which is about to get deprecated) called Networks.
Networks are created for schools, universities and other big organizations. They had the name of the high school so they have to list it as their own high school in their Facebook profiles.
This will automatically put them in the same network as the girl. I am assuming here that she has listed that on her Facebook profile.
Then performing a graph search for women who has gone to that high school and live in Austin with an age between X and Y will narrow the search a lot. Even if some of that information is not public on Facebook it would be public to them if they are not same network if she has the default settings. This is a neat trick which is still usable.
I don't think looking through the pictures would take more than half an hour.
Is it just me or is this spectacularly creepy? "I bet you can't cyber stalk my friend! I bet you we can!" I could understand if this was at defcon or a security/privacy related conference and all parties involved were in the field, but just cyber stalking some random woman? What's the point?
Assuming that she didn't show up in a reverse Google Image Search, maybe they used some sort of facial recognition API (like http://www.lambdal.com/), perhaps along with the other people in the picture, against scraped pictures from LinkedIn and Facebook based on her appx age, location (Austin), gender (F), and high school.
I think the key is knowing what high school she went to. From there, you could possibly get yearbook pictures and student lists. Knowing an approximate age would narrow the possibilities significantly as well.
So basically the engineers knew something specific about the girl already which narrowed their data set. Unless they tell you what that was, we can go all day long guessing how they "figured" it out. In fact with this information I am interested in the actual time taken to find the girl. They could have found her in five minutes after receiving the picture and you wouldn't know. Why? From the article:
1) They were running software on a computer for hours
They are engineers!
2) The first couple of tries didn’t work
Are you sure they weren't debugging?
3) They knew what high school she went too prior to searching
If they know the high school they only have to find one or two established teachers who were around for a long period of time. It is not unlikely they can call a name from a picture after all those years.
This smells like viral marketing to me (too bad I still don't know what close.io is, should have added some information to the article. Or are you looking for new engineers? "Hey look! At close.io we seal bets with engineers. About girls! Come work for us!").
Possibilities:
- they looked at signups at close.io after the evening, thus narrowing a list of female names for image searching. They could further limit this to IP's from the austin area.
- They bought the ad words 'justin gold', and 'justinbgold'. Then waited for her to google you, thus revealing IP addresses for further googling.
Saying she "doesn't exist on the internet" is a big assumption. First, she is on Facebook, so she's on the internet, easily accessible or no. Second, most people leave some internet trail through online forums, newspaper articles and newsletters.
I find it very likely that she was found on the internet somewhere.
I'm guessing it has to be something to do with facial recognition, since he asked for the picture without knowing its contents (other than that it has her face in it).
From there, if he had an idea of where they went that night, a club or bar, perhaps he could do some Facebook/Twitter searching magic to search for pictures of that night, taken by other people. Then they could look for her face in the background.
Not sure where to go from there... perhaps they got lucky with one of the pictures being tagged with name. Or perhaps they identified everyone she was shown to be associating with, and found intersections in their Facebook friends list? Does it have to do with Facebook?
How are you surprised that they did it when they knew what high school she went to? Two possibilities:
Option 1: Step 1) Find copies of yearbooks going back a few years, 2) find her face in the yearbook.
Option 2: Step 1) Search facebook for people who went to the same high school as she did in the probable time frame. 2) Look for her in that set, or in the set of friends that the people who matched the search have.
Feel free to sprinkle in face detection on the yearbook/facebook photos to make it more engineer-ey.
As a matter of fact, how did they find out what high school she went to? I can't imagine you telling them that if you were not providing much information about her to anyone else.
[+] [-] philfreo|12 years ago|reply
- This isn't the actual photo but looks similar, except that she was alone. Justin was afraid of reposting the same photo after the results of handing over a photo last time :)
- There was no useful exif data in the image.
- We knew her high school and approximate age in Austin, but were unable to find any yearbooks / class rosters online. (Idea not pursued: try to acquire a yearbook offline).
- Facebook Graph Search and LinkedIn with the criteria we knew didn't help.
- Google Image search didn't help.
> "I'm sure there's a lot of information missing from this story"
Quite true.
Ultimately we had several methods going at once to try to figure it out.
The one that came back with results first relies on some information Justin still doesn't know we had (though he would still be impressed), but other approaches we also expected to work would be possible with only what's in the story, and haven't yet been mentioned here.
EDIT: shared more at https://news.ycombinator.com/item?id=6369751
[+] [-] grecy|12 years ago|reply
* Everyone was at the original bar at the same time - someone else had photos from the night that had her and other people in them, and those people (her friends) could be found online.
* The bar had some kind of "sign-up for x" list, or some kind of contest, or some kind of photos of the night, which were helpful.
* You searched twitter/facebook/whatever for a girl posting about a boy she just met at bar X, blah blah
* We know she was going to some kind of event - you guys figured out what that was, and found a guest list or something.
* You guys did a some wi-fi sniffing and read a few emails. He did say he emailed you the photo, so you know which account to listen for.
Are any of those close?
[+] [-] evan_|12 years ago|reply
Or maybe you knew that she wanted him to meet her at an event and you found a roster for that event- or ie if it was a real estate convention you would have a pretty good idea that she was a realtor and could look at websites for realtors in Austin.
Or maybe you checked his recent facebook friends, looked at his recent twitter followers, and everyone who liked/commented on his recent photos/posts on Instagram and whatever other social networks he's on- as well as the photos/posts he liked/commented on.
I think you took a (relatively) low-tech approach to this.
[+] [-] jpmc|12 years ago|reply
Find the event Thursday night at 8:00pm in Austin and start tracking down from there.
[+] [-] mmmooo|12 years ago|reply
[+] [-] JesseAldridge|12 years ago|reply
[+] [-] pmorici|12 years ago|reply
The second photo you realized you had showed the subject holding the device used to take the picture sent to your colleague. You investigated that particular device and found a way to uniquely identify it's pictures based off some characteristic of it's photos, not necessarily EXIF, then scoured the Internet for photos with the same unique characteristic until you found one that revealed more information about the subject?
[+] [-] niuzeta|12 years ago|reply
It is also possible that during the few days' correspondence one of the engineer guys had some other piece of info that we are not familiar with. Seeing how the engineers already knew her high school, I'm sure Justin talked about her with co-workers.
I think the key information that's missing here is what the engineers knew but Justin didn't know that they knew.(Sounds like a quote out of a Friends episode, mmm)
[+] [-] elorant|12 years ago|reply
[+] [-] wahsd|12 years ago|reply
[+] [-] joetann|12 years ago|reply
[+] [-] nathas|12 years ago|reply
At that point, if the bartender knows some of her backstory or friends or anything - maybe only has a first name - things get much simpler.
[+] [-] fnordfnordfnord|12 years ago|reply
[+] [-] fnordfnordfnord|12 years ago|reply
[+] [-] jasdeepsingh|12 years ago|reply
[+] [-] abdelix|12 years ago|reply
[+] [-] zzzcpan|12 years ago|reply
[+] [-] zzzcpan|12 years ago|reply
[+] [-] antr|12 years ago|reply
[+] [-] dkb|12 years ago|reply
[deleted]
[+] [-] espeed|12 years ago|reply
[deleted]
[+] [-] rorrr2|12 years ago|reply
[+] [-] unknown|12 years ago|reply
[deleted]
[+] [-] evan_|12 years ago|reply
Simplest explanation: He knew before he made the bet.
[+] [-] philfreo|12 years ago|reply
Of course one of the first things I did was ask another friend who I thought met her. But no, ultimately the answer did come from the internet, not from anyone I knew.
[+] [-] abalone|12 years ago|reply
Advice to woman: Run.
Advice to startups: Don't let your sales and engineering teams be giant douche canoes and if they are certainly don't publicly brag about it.
[+] [-] philfreo|12 years ago|reply
- This was just a fun bet between friends. Justin really didn't think it could be done and I wanted to show him how a little information goes a long way online.
- Justin treated this girl well and she even knew about this bet.
- "Look up online" != Stalking
- None of the discovered information has been posted publicly online. This isn't even her photo.
[+] [-] bunkat|12 years ago|reply
[+] [-] jlees|12 years ago|reply
[+] [-] sergiotapia|12 years ago|reply
[+] [-] socialist_coder|12 years ago|reply
Why would this post be hosted on the close.io blog and not on a personal blog? It has nothing to do with close.io.
Why would he mention close.io so many times?
Why would he ask about the quality of the "close.io engineers" instead of just his coworker or friend?
Smells fishy to me.
[+] [-] swamp40|12 years ago|reply
That's only a couple hundred kids, each with a couple hundred friends.
If you still get nothing, start doing friend requests for the people that don't show their friends. I'll bet you'd get another 25-30 kids that way.
I don't see why you'd need facial recognition software to go thru 20,000 pictures.
3/4ths of them would be the wrong gender/age and be immediately disqualified.
But if you're a programmer, you can probably scrape and aggregate to speed things up - drop out the males, other high schools, only show pictures once, etc.
[+] [-] saalweachter|12 years ago|reply
So you've found an attack vector where you can get a woman's name and address off the 'net with a picture and a small amount of information. Maybe it doesn't work all the time, but it worked at least once. What are you going to do with that attack vector? Are you going to make an iPhone app, so that others can snap pictures of random women and recreate your exploit automatically? Or are you going to come up with ways the attack vector can be shut down, things that either individuals (eg, the woman herself) or organizations (like Facebook or Google) can do to block the attack?
[+] [-] cbhl|12 years ago|reply
So, here's a thought. Let's say the Engineer has access to the SMS logs (NSA-style) for Justin's phone. (Maybe his phone goes through their system. But even if not, maybe it's company-provided, so there's an admin interface provided by the telecom that lists this information.)
Justin probably doesn't regularly text people in Texas, since the team is based out of Palo Alto. So look through his logs for a number in a Texas area code (http://en.wikipedia.org/wiki/List_of_Texas_area_codes) that he's been texting more often than usual.
That gets you the girl's phone number; probably a cell phone. Now, you need to go from the cell phone number to a last name. Reverse Phone Lookup probably won't work for this, although it's certainly worth a first try. More likely candidates: looking through the users table at close.io for a matching phone number, or doing a google search for the phone number. Or, try calling in the middle of the night and seeing what the voicemail says. Any of these approaches might work.
Once you've got a last name, grab a copy of the white pages. Most public schools publish their district boundaries, so go grab a copy of that. Look through all entries that match the last name, and see how many of them fall within (or close to, in the case politics made the boundary change) the district boundary. You probably will get a handful (maybe three or four). If you only get one, bingo!
If you get more than one, I'd call the cell phone to see if I can get a first name somehow. Then, I'd go through the landlines in the White Pages and call them one at a time, "Hello, may I speak to X? Sorry, wrong number." until one matches.
[+] [-] gingergirl|12 years ago|reply
[+] [-] sliverstorm|12 years ago|reply
[+] [-] VLM|12 years ago|reply
Note the convoluted written responses about not talking to anyone he knew and so forth which technically does not exclude the girl, herself, saying her own name.
The other alternative is she's somebody's buddy/family member so it would be trivial to know before hand that she's so and so's ex or sister or whatever and given that intel, figure it out.
[+] [-] hkdobrev|12 years ago|reply
I assume the author is not a Facebook friend with the girl. Otherwise it would have been too easy. But friends of friends is not the only privacy leak on Facebook. There is a very old feature (which is about to get deprecated) called Networks.
Networks are created for schools, universities and other big organizations. They had the name of the high school so they have to list it as their own high school in their Facebook profiles.
This will automatically put them in the same network as the girl. I am assuming here that she has listed that on her Facebook profile.
Then performing a graph search for women who has gone to that high school and live in Austin with an age between X and Y will narrow the search a lot. Even if some of that information is not public on Facebook it would be public to them if they are not same network if she has the default settings. This is a neat trick which is still usable.
I don't think looking through the pictures would take more than half an hour.
[+] [-] michaelmior|12 years ago|reply
"- Facebook Graph Search and LinkedIn with the criteria we knew didn't help."
https://news.ycombinator.com/item?id=6369519
[+] [-] jmccree|12 years ago|reply
[+] [-] ngoel36|12 years ago|reply
At least that's what I might have done.
[+] [-] jbigelow76|12 years ago|reply
[+] [-] qu4z-2|12 years ago|reply
[+] [-] sp332|12 years ago|reply
[+] [-] mrspandex|12 years ago|reply
[+] [-] pearjuice|12 years ago|reply
1) They were running software on a computer for hours
They are engineers!
2) The first couple of tries didn’t work
Are you sure they weren't debugging?
3) They knew what high school she went too prior to searching
If they know the high school they only have to find one or two established teachers who were around for a long period of time. It is not unlikely they can call a name from a picture after all those years.
This smells like viral marketing to me (too bad I still don't know what close.io is, should have added some information to the article. Or are you looking for new engineers? "Hey look! At close.io we seal bets with engineers. About girls! Come work for us!").
[+] [-] b_emery|12 years ago|reply
[+] [-] scott_s|12 years ago|reply
I find it very likely that she was found on the internet somewhere.
[+] [-] lifeformed|12 years ago|reply
From there, if he had an idea of where they went that night, a club or bar, perhaps he could do some Facebook/Twitter searching magic to search for pictures of that night, taken by other people. Then they could look for her face in the background.
Not sure where to go from there... perhaps they got lucky with one of the pictures being tagged with name. Or perhaps they identified everyone she was shown to be associating with, and found intersections in their Facebook friends list? Does it have to do with Facebook?
[+] [-] oh_sigh|12 years ago|reply
Option 1: Step 1) Find copies of yearbooks going back a few years, 2) find her face in the yearbook.
Option 2: Step 1) Search facebook for people who went to the same high school as she did in the probable time frame. 2) Look for her in that set, or in the set of friends that the people who matched the search have.
Feel free to sprinkle in face detection on the yearbook/facebook photos to make it more engineer-ey.
As a matter of fact, how did they find out what high school she went to? I can't imagine you telling them that if you were not providing much information about her to anyone else.