Not just facial recognition, the "terror watch list" failed (after a warning from Russia about the older brother) lack of background checks on the AR15s they owned failed, the secured perimeter failed (the teenager was outside of it) the city lockdown failed (the guy smoking the cigarette outside when he saw the blood was technically violating the lockdown).
In the end it was mostly dumb luck, dumb criminals, and the fact the poor victim who lost their legs managed to survive. In fact that was the success, that more wounded people didn't die thanks to fantastic work by the hospital and doctors.
Instead of patting law enforcement on their backs, how about thanking the doctors, where is their applause?
It seems disingenuous to say that facial recognition failed. As far as I can tell, all the images we had of these suspects were either very low-resolution, taken from angles, or both. Seems plausible that law enforcement didn't have any good candidate images from which to run a facial recognition search.
> In the end it was mostly dumb luck
What? Painstaking police work went in to getting images of these suspects, and the release of those photos are almost certainly what led to Thursday's events and, ultimately, the killing and capture of the suspects.
> the city lockdown failed (the guy smoking the cigarette outside when he saw the blood was technically violating the lockdown)
Firstly, the lockdown is what forced suspect #2 to hide out in a boat instead of fleeing the area. Secondly, and less importantly, the guy smoking a cigarette wasn't violating the lockdown. It had ended prior to him going outside.
Was there luck involved? Of course. But there was also tremendous work done by a number of law enforcement agencies - without which, the suspects may never have been killed and captured.
that more wounded people didn't die thanks to fantastic work by the hospital and doctors
The EMTs and hospitals did marvelous work, but one nonobvious fact that helped a great deal was that the bombing was within 2 miles of FIVE DIFFERENT level 1 trauma centers. All five are world class teaching hospitals and one of them, MGH, had been recognized as the best hospital in the US at one point.
There are several states that don't have even a single level 1 trauma center....
Actually, the lockdown was voluntary, though with strong community pressure to comply.
But yes, multiple subsystems failed in the police response. If you listened to police scanners, you also heard that they had some trouble identifying streets correctly, or keeping their mics off when not talking. But police response protocols are one human system that definitely needs to be able to work in the face of multiple subsystem failure. I say judge the police on how the response worked as a whole.
I have to say though, thank god that Dunkin' Doughnuts' Tactical Ration Deployment Vehicle ran smoothly.
What you fail to realize is that the "terror watch list" you speak of doesn't hold people on it from just any hearsay. The articles said that the FBI did its due diligence and found no reason to continue an investigation on them.
Think about it this way: would you want just anyone to be watchlisted based on tips that could be politically motivated, intentionally misleading, or just completely factually inaccurate?
What you can never account for in terrorism screening is the lone-wolf element. There will always be some fringe guys in parts of the world who hatch their plots outside of the view of what we're able to observe. The only options you have are to limit their access to harmful weapons/devices, or contain them as fast as possible when they do go on the offensive.
Yes, doctors and other emergency responders did a great job, but don't be so quick to take away a victory from law enforcement when there are very few positive stories circulated about them ever. The boston marathon runners/crowd didn't know they were putting their lives on the line that day. Law enforcement, on the other hand, made a conscious decision to do so.
We don't yet know if the "terror watch list" failed. Perhaps he was checked and they found nothing, because at that time, they weren't terrorists and weren't planning a terrorist attact at that point? We're not living in a minority-report style pre-crime detection world yet.
this is absurd. i'm in Boston. there is absolutely no shortage of support and praise for the doctors and hospitals here.
lots of things went wrong for this to happen. thanks for pointing out the painfully obvious. our police deserve the "pats on the back" (which, by the way, is an extremely condescending way to put it) they're getting, and more.
Massachusetts does in fact have an assault weapon ban in place (similar to the 1994 Federal AWB), although you can get modified/MA-compliant versions of many "assault weapons" such as the AR-15/AK that are legal, however you are limited to a 10 round magazine. They would have needed a Class-A LTC to even contemplate owning such a weapon -- this involves fingerprints, a full background check, references and a "letter of purpose" along with permission from your local police chief to get.
However, the younger brother was too young to get a permit, the older brother was in fact not a US citizen and was here on a green card and thus is ineligible for a gun permit. Having said that, even if he was a US citizen his previous arrest for domestic assault would've rendered him ineligible for/and or revoked an existing LTC-A had he had one.
I have not seen anything yet reported as to what kind of long gun(s) they had or where they got them. The FBI warned agents responding to Watertown on Thursday night that they had "fully automatic AK47s" [along with grenades/dynamite and IEDs]. It is my understanding that they only recovered two handguns at the scene of the firefight in Watertown and one handgun and a long gun were found on the boat after the final surrender [aside from the IEDs scattered over Watertown].
It is interesting that despite the incredible firepower the brothers had access to (handguns, long gun(s), multiple IEDs), that when there's someone shooting back, the brothers plans were pretty much shut down and the loss of life/injury was very minimal considering the circumstances.
The problem is that any non-perfect automated system will wrongly convict so many civilians as being terrorists that nobody will trust it anymore.
Let's say that you want to catch terrorists creeping around in your airport. To do this, you have a magic camera that, when it takes a picture of a civilian, compares it to a picture of a known terrorist. If they're in fact the same person, it will flag security over. Let's say that it is indeed a magic camera and makes the correct decision 99.99% of the time, which is far better than the current state of the art[1].
Do you have any idea how much DHS or DoD would pay for such a magic camera?
But it won't be enough.
Consider a watch list of 100 terrorists that we're looking for in our airport. If our camera compares each civilian's picture to all 100 terrorists in succession, and if we model each comparison as a Bernoulli trial, then each civilian has a 0.9999^100 = 99.004% chance of getting through security unfazed, which means we flag innocent people as terrorists about 1% of the time.
This doesn't work in an airport setting. Assuming 5,000 people pass through your small airport per hour and you have a 14-hour day, that means your security people have to go through 0.01 * 5000 * 14 = 700 false alarms each day. That's about 1.2 false alarms per minute. Your security people will have to go through all their checks each time this happens. Before long, they're going to stop trusting your magic camera.
As you scale your watch list up, it gets worse. How well would it work for 1,000 people on the watch list? 10,000? Note that the current US watch list has 50,000 people on it.
Even if it were a perfect magic camera, it coudln't catch terrorists on their maiden crime who aren't on the watch list at all.
Do you know how unreliably people recognize strangers? Many people are already wrongly convicted on such evidence.
Even perfect automated facial recognition won't convict people. It will merely point out people that should be looked at. If a jury looking at the image and the guy's face think the system is wrong, they can disregard it.
As far as prevention goes, what we need are systems that can detect deception reliably, which will help if we ever decide to ask people meaningful questions. Not to convict, but to focus attention on a possible threat.
This is a completely different situation; there are 2 people we have pictures of that we know are terrorists; we have a database of suspected terrorists; had we found 100 matches in that database, with one of them being the right one, it would have narrowed the investigation considerably. It was the perfect situation for AFR, had AFR either been more advanced, or the pictures been more amenable to current AFR technology.
I am the author of the referenced article. The problem is matching surveillance camera images (like from Lord & Taylor's) is impossible because of the steep down tilt angle of the cameras plus their low resolution. Smartphone images could be a little better as the angle is more head on and the resolution is higher. However, even those images tended to be in a wide field of view which means low pixel density, hurting matching performance. Last, but not least, the caps and sunglasses hurt recognition as well.
Matching surveillance camera images is always possible no matter how bad the image quality is. The only matter is how many false positives show up.
Facial recognition algorithms always give their best matching candidates, with thresholds to skip those not quite matching ones. Your article makes it sound like no such results were produced at all, which is unlikely how recognition is supposed to work.
How hard would it be to throw off facial recognition by simply wearing very high quality makeup and prosthetics to alter your facial appearance (ala hollywood movies)? If I were the boston bombers, I would have first learned how to make myself look completely different. It's unbelievable to me that these guys didn't realize they'd be captured on tons of cameras that day.
If you wore Mission Impossible style masks, there's no reason why the facial recognition would be able to pick you up from underneath. You might alert the suspicion of people around you more though, as they don't ever seem to be perfect.
When the FBI released the photos of the two suspects, I kept wondering, "Why do they need the publics help when I'm sure there is facial recognition software to help match the suspects' faces?" Well I guess this article somewhat answers this, but I am still in disbelief that our government does not have better software.
For how long it took to discover their names, the two suspects had TOO much time to escape. They had about 48 hours to book flights out of the country, flee to another part of the country, do more damage, etc. The lack of facial recognition software could have ultimately led to this case being unnecessarily longer. This is an issue that CAN and NEEDS to be fixed quickly--in fact, I hope the government is working on a more sophisticated facial recognition software right now. Ideally, the facial recognition software should be able to crawl through the web to search for faces.
Now that I think about it, a good hacker could build a search engine that allows people to post photos and the program will run an automated search across the web (Social networks, Google, etc.) to see which pictures match it. Hell, Facebook even has facial recognition to some extent. A more sophisticated one cannot be that difficult for the government to build and put to use.
There is a lot of money being put into facial recognition software by law enforcement and the military (including DARPA), so it's certainly the case that it's being worked on. But it's a hard problem, especially to do robustly at a large scale. What they do have is almost certainly the state of the art— in fact much of the current commercial state-of-the-art is itself based on stuff developed in older DARPA projects.
Facebook only has to distinguish among friends of the uploader. And it's not perfect. This man was wearing sunglasses and a baseball cap. He would not have been identifiable out of the tens of thousands of people at the event, matched across the (probably) millions of people in the database.
What's wrong with needing the public's help? The public did a great job in providing critical evidence.
Maybe instead of more intrusive surveillance we find more ways for the public to participate. Maybe we uphold the public's right to take pictures and video in "sensitive areas". Etc.
I get what you mean when you say these guys had way too much time to make some kind of an escape and I agree then that there is an absolute need for a much better software. Having said that, I do not think it would be wise for government to build and use an "image crawler" to search and store images of citizens...it would provoke widespread criticism concerning privacy of millions of citizens. On top of that in this particular instance, the police commissioner points out that "both Tsarnaevs’ images exist in official databases"..the software failed to make a match.
I watched this lecture recently http://youtu.be/-IUe3ce_29I which talked about how hard it is to do image recognition at low resolution. One solution is a "fovea" system that scans an area for interesting items and zooms in on them with a high resolution camera automatically to get better recognition.
I wonder if this would help in similar instances. A system that uses facial detection to isolate faces and then zoom in on them very quickly seems feasible. I'm assuming it would be overwhelmed in a scenario like a marathon of course, but perhaps it would help in other situations.
My big questions is why were the suspects faces in the database? Why would a 19 year old, never done anything face be in the facial recognition database?
I'm going to venture to guess that either a) the officer was talking out of his ass to make a quick point or b) someone, somewhere massively scraped every social network they could think of for anyone with ties to the area or the event. That being said, I don't really like the sound of it either, regardless of how true it is.
--pre-submit thought: perhaps it was the older brother? Our background is rather limited but it would not be surprising if he had had prior run-ins with the law. Even a minor offense will get your mugshot in the file.
Facial recognition is still treated like some magical technology, like the "zoom in, enhance" feature much mocked about on TV crime shows. I bet Facebook's technology is ahead of law enforcement's in terms of data, algorithm, and refinement, and yet it still stumbles on pretty obvious faces (if you have the default auto-tag feature left on)
I can understand that recording live footage with very high quality is a problem, but how about capturing quality stills to complement the video with lower quality.
Someone with knowledge of cameras please either correct or endorse this, but I think the largest price/quality tradeoff in any camera happens at the lens choice, and given a cheap lens, the stills can't be much better than the video.
Bottom line is that it's nearly impossible to exactly match facial features to anyone beyond a set distance when there's too much ambiguity between distances and rubbish quality CCTV video.
Matching gait may be a more effective means of identification at a distance (and, in this and many other cases, at odd angles).
There are a few, though less than you might think.
Lots of general progress has been made in terms of facial detection algorithms, but there are almost no "general" security camera deployments that are setup to capture sufficient detail to get a good face match.
There are not enough events like this in open places that drive requirements for facial recognition use. Limited cases (airports, casinos, etc.) more frequently have cameras setup to get good straight-on high-res facial shots, but almost no typical outdoor area is covered that way.
[+] [-] ck2|13 years ago|reply
In the end it was mostly dumb luck, dumb criminals, and the fact the poor victim who lost their legs managed to survive. In fact that was the success, that more wounded people didn't die thanks to fantastic work by the hospital and doctors.
Instead of patting law enforcement on their backs, how about thanking the doctors, where is their applause?
[+] [-] kevincennis|13 years ago|reply
> In the end it was mostly dumb luck
What? Painstaking police work went in to getting images of these suspects, and the release of those photos are almost certainly what led to Thursday's events and, ultimately, the killing and capture of the suspects.
> the city lockdown failed (the guy smoking the cigarette outside when he saw the blood was technically violating the lockdown)
Firstly, the lockdown is what forced suspect #2 to hide out in a boat instead of fleeing the area. Secondly, and less importantly, the guy smoking a cigarette wasn't violating the lockdown. It had ended prior to him going outside.
Was there luck involved? Of course. But there was also tremendous work done by a number of law enforcement agencies - without which, the suspects may never have been killed and captured.
[+] [-] MichaelSalib|13 years ago|reply
The EMTs and hospitals did marvelous work, but one nonobvious fact that helped a great deal was that the bombing was within 2 miles of FIVE DIFFERENT level 1 trauma centers. All five are world class teaching hospitals and one of them, MGH, had been recognized as the best hospital in the US at one point.
There are several states that don't have even a single level 1 trauma center....
[+] [-] afarrell|13 years ago|reply
But yes, multiple subsystems failed in the police response. If you listened to police scanners, you also heard that they had some trouble identifying streets correctly, or keeping their mics off when not talking. But police response protocols are one human system that definitely needs to be able to work in the face of multiple subsystem failure. I say judge the police on how the response worked as a whole.
I have to say though, thank god that Dunkin' Doughnuts' Tactical Ration Deployment Vehicle ran smoothly.
[+] [-] bazillion|13 years ago|reply
Think about it this way: would you want just anyone to be watchlisted based on tips that could be politically motivated, intentionally misleading, or just completely factually inaccurate?
What you can never account for in terrorism screening is the lone-wolf element. There will always be some fringe guys in parts of the world who hatch their plots outside of the view of what we're able to observe. The only options you have are to limit their access to harmful weapons/devices, or contain them as fast as possible when they do go on the offensive.
Yes, doctors and other emergency responders did a great job, but don't be so quick to take away a victory from law enforcement when there are very few positive stories circulated about them ever. The boston marathon runners/crowd didn't know they were putting their lives on the line that day. Law enforcement, on the other hand, made a conscious decision to do so.
[+] [-] joosters|13 years ago|reply
[+] [-] localhost3000|13 years ago|reply
lots of things went wrong for this to happen. thanks for pointing out the painfully obvious. our police deserve the "pats on the back" (which, by the way, is an extremely condescending way to put it) they're getting, and more.
[+] [-] unvs|13 years ago|reply
[+] [-] Anechoic|13 years ago|reply
But for the shelter-in-place order (it wasn't a lockdown or curfew) the guy may not have been home in the first place for find Tsarnaev.
[+] [-] lsllc|13 years ago|reply
http://www.boston.com/news/local/massachusetts/2013/04/21/ma...
Massachusetts does in fact have an assault weapon ban in place (similar to the 1994 Federal AWB), although you can get modified/MA-compliant versions of many "assault weapons" such as the AR-15/AK that are legal, however you are limited to a 10 round magazine. They would have needed a Class-A LTC to even contemplate owning such a weapon -- this involves fingerprints, a full background check, references and a "letter of purpose" along with permission from your local police chief to get.
However, the younger brother was too young to get a permit, the older brother was in fact not a US citizen and was here on a green card and thus is ineligible for a gun permit. Having said that, even if he was a US citizen his previous arrest for domestic assault would've rendered him ineligible for/and or revoked an existing LTC-A had he had one.
I have not seen anything yet reported as to what kind of long gun(s) they had or where they got them. The FBI warned agents responding to Watertown on Thursday night that they had "fully automatic AK47s" [along with grenades/dynamite and IEDs]. It is my understanding that they only recovered two handguns at the scene of the firefight in Watertown and one handgun and a long gun were found on the boat after the final surrender [aside from the IEDs scattered over Watertown].
It is interesting that despite the incredible firepower the brothers had access to (handguns, long gun(s), multiple IEDs), that when there's someone shooting back, the brothers plans were pretty much shut down and the loss of life/injury was very minimal considering the circumstances.
[+] [-] logn|13 years ago|reply
I guess when you create a panopticon you're relying more on psychology than surveillance.
[+] [-] darkarmani|13 years ago|reply
Flamebait. Anything people point out, you just define as luck. Luck that LEO found evidence, had pictures of the perps, etc.
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] gcr|13 years ago|reply
Let's say that you want to catch terrorists creeping around in your airport. To do this, you have a magic camera that, when it takes a picture of a civilian, compares it to a picture of a known terrorist. If they're in fact the same person, it will flag security over. Let's say that it is indeed a magic camera and makes the correct decision 99.99% of the time, which is far better than the current state of the art[1].
Do you have any idea how much DHS or DoD would pay for such a magic camera?
But it won't be enough.
Consider a watch list of 100 terrorists that we're looking for in our airport. If our camera compares each civilian's picture to all 100 terrorists in succession, and if we model each comparison as a Bernoulli trial, then each civilian has a 0.9999^100 = 99.004% chance of getting through security unfazed, which means we flag innocent people as terrorists about 1% of the time.
This doesn't work in an airport setting. Assuming 5,000 people pass through your small airport per hour and you have a 14-hour day, that means your security people have to go through 0.01 * 5000 * 14 = 700 false alarms each day. That's about 1.2 false alarms per minute. Your security people will have to go through all their checks each time this happens. Before long, they're going to stop trusting your magic camera.
As you scale your watch list up, it gets worse. How well would it work for 1,000 people on the watch list? 10,000? Note that the current US watch list has 50,000 people on it.
Even if it were a perfect magic camera, it coudln't catch terrorists on their maiden crime who aren't on the watch list at all.
(Most of this post was adapted from: http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt , which is a really interesting presentation if you're interested in these sorts of things.)
1: http://vis-www.cs.umass.edu/lfw/results.html
[+] [-] stretchwithme|13 years ago|reply
Even perfect automated facial recognition won't convict people. It will merely point out people that should be looked at. If a jury looking at the image and the guy's face think the system is wrong, they can disregard it.
As far as prevention goes, what we need are systems that can detect deception reliably, which will help if we ever decide to ask people meaningful questions. Not to convict, but to focus attention on a possible threat.
[+] [-] aidenn0|13 years ago|reply
[+] [-] jhonovich|13 years ago|reply
[+] [-] xfs|13 years ago|reply
Facial recognition algorithms always give their best matching candidates, with thresholds to skip those not quite matching ones. Your article makes it sound like no such results were produced at all, which is unlikely how recognition is supposed to work.
[+] [-] marknutter|13 years ago|reply
[+] [-] nwh|13 years ago|reply
[+] [-] speednoise|13 years ago|reply
[+] [-] Plimsoll|13 years ago|reply
Don't know how easy it is to get those masks though.
[+] [-] dyno12345|13 years ago|reply
Costs $3, will beat any facial recognition
[+] [-] gcr|13 years ago|reply
But then again, the police will label you as "that guy with weird bits of plastic on his face", which won't help you hide at all.
[+] [-] openmx|13 years ago|reply
[+] [-] aashaykumar92|13 years ago|reply
For how long it took to discover their names, the two suspects had TOO much time to escape. They had about 48 hours to book flights out of the country, flee to another part of the country, do more damage, etc. The lack of facial recognition software could have ultimately led to this case being unnecessarily longer. This is an issue that CAN and NEEDS to be fixed quickly--in fact, I hope the government is working on a more sophisticated facial recognition software right now. Ideally, the facial recognition software should be able to crawl through the web to search for faces.
Now that I think about it, a good hacker could build a search engine that allows people to post photos and the program will run an automated search across the web (Social networks, Google, etc.) to see which pictures match it. Hell, Facebook even has facial recognition to some extent. A more sophisticated one cannot be that difficult for the government to build and put to use.
[+] [-] mjn|13 years ago|reply
[+] [-] sp332|13 years ago|reply
[+] [-] sehugg|13 years ago|reply
Maybe instead of more intrusive surveillance we find more ways for the public to participate. Maybe we uphold the public's right to take pictures and video in "sensitive areas". Etc.
[+] [-] sy_r0y|13 years ago|reply
[+] [-] jere|13 years ago|reply
I wonder if this would help in similar instances. A system that uses facial detection to isolate faces and then zoom in on them very quickly seems feasible. I'm assuming it would be overwhelmed in a scenario like a marathon of course, but perhaps it would help in other situations.
[+] [-] bsenftner|13 years ago|reply
[+] [-] UnoriginalGuy|13 years ago|reply
Everyone who applies for a US visa is additionally photographed and fingerprinted.
Everyone who applies for naturalisation is additionally photographed and fingerprinted.
The suspects did all three of these things. The US government has almost every foreigner legally on American soil in a database.
Plus then you have the DNV on top of that and or passport office.
[+] [-] dr_doom|13 years ago|reply
http://www.cbsnews.com/8301-201_162-57580534/fbi-interviewed...
[+] [-] sp332|13 years ago|reply
[+] [-] DannyBee|13 years ago|reply
[+] [-] ics|13 years ago|reply
--pre-submit thought: perhaps it was the older brother? Our background is rather limited but it would not be surprising if he had had prior run-ins with the law. Even a minor offense will get your mugshot in the file.
[+] [-] astar|13 years ago|reply
[+] [-] brk|13 years ago|reply
The average cctv shot will have around 200-1000 pixels on the entire face, if you're lucky.
[+] [-] gcr|13 years ago|reply
US law enforcement has to distinguish between 300 million civilians.
Of the two, I would expect Facebook to be more successful at identifying peolpe.
[+] [-] anigbrowl|13 years ago|reply
Actually, those jokes refer to a scene in Blade Runner: https://www.youtube.com/watch?v=qHepKd38pr0
[+] [-] jpalomaki|13 years ago|reply
[+] [-] emiliobumachar|13 years ago|reply
[+] [-] mikecane|13 years ago|reply
[+] [-] anigbrowl|13 years ago|reply
[+] [-] eksith|13 years ago|reply
Matching gait may be a more effective means of identification at a distance (and, in this and many other cases, at odd angles).
http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf (PDF)
[+] [-] mrkmcknz|13 years ago|reply
I know of Lambda labs but I personally haven't use their solution post Face.com.
[+] [-] brk|13 years ago|reply
There are not enough events like this in open places that drive requirements for facial recognition use. Limited cases (airports, casinos, etc.) more frequently have cameras setup to get good straight-on high-res facial shots, but almost no typical outdoor area is covered that way.
[+] [-] WestCoastJustin|13 years ago|reply
[+] [-] new_test|13 years ago|reply