UK Police Say 92 Percent False Positive Facial Recognition Is No Big Deal (arstechnica.com) 189
An anonymous reader quotes a report from Ars Technica: A British police agency is defending its use of facial recognition technology at the June 2017 Champions League soccer final in Cardiff, Wales -- among several other instances -- saying that despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error. New data about the South Wales Police's use of the technology obtained by Wired UK and The Guardian through a public records request shows that of the 2,470 alerts from the facial recognition system, 2,297 were false positives. In other words, nine out of 10 times, the system erroneously flagged someone as being suspicious or worthy of arrest.
In a public statement, the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts over the last nine months. "Of course, no facial recognition system is 100 percent accurate under all conditions. Technical issues are normal to all face recognition systems, which means false positives will continue to be a common problem for the foreseeable future," the police wrote. "However, since we introduced the facial recognition technology, no individual has been arrested where a false positive alert has led to an intervention and no members of the public have complained." The agency added that it is "very cognizant of concerns about privacy, and we have built in checks and balances into our methodology to make sure our approach is justified and balanced."
In a public statement, the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts over the last nine months. "Of course, no facial recognition system is 100 percent accurate under all conditions. Technical issues are normal to all face recognition systems, which means false positives will continue to be a common problem for the foreseeable future," the police wrote. "However, since we introduced the facial recognition technology, no individual has been arrested where a false positive alert has led to an intervention and no members of the public have complained." The agency added that it is "very cognizant of concerns about privacy, and we have built in checks and balances into our methodology to make sure our approach is justified and balanced."
Let's be positive (Score:5, Funny)
Re:Let's be positive (Score:5, Insightful)
Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.
Indeed. If you are looking for a suspect in a city of a million people, and this system flags 10 people, and upon double checking you find that one of the ten is the suspect, then that is pretty darn good.
The false positive rate, by itself, tells you nothing about the usefulness of a test.
Re: (Score:1)
Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.
Indeed. If you are looking for a suspect in a city of a million people, and this system flags 10 people, and upon double checking you find that one of the ten is the suspect, then that is pretty darn good.
The false positive rate, by itself, tells you nothing about the usefulness of a test.
The real problem is that most violent criminals are black and facial recognition has a harder time with black faces because of lower contrast. Check the crime stats for the US and any other nation with a significant black minority population. Blacks commit violent crimes far in excess of their percentage of the population. This includes nations like Sweden which never had slavery or Jim Crow. These are facts.
Re: (Score:2)
The real problem is that most violent criminals are black and facial recognition has a harder time with black faces because of lower contrast.
I'm pretty sure that a lot of people will be happy with this anti-profiling affirmative action.
Re: (Score:2)
Also, British politics is extremely PC, so burkas are allowed. Which means criminals just need to dress up in a black tent and nobody can make them show their face.
Re: (Score:2)
Re: (Score:2)
No let's because it isn't facts and most likely based on racist propaganda (alternatively the AC being a troll/idiot).
I need only one point to prove this: Most violent criminals in Sweden are not blacks.
Making that claim means the AC have fallen for falsehoods propagated by racist and cryptoracist groups. Those that also like to claim "no go" zones in Sweden - another obvious lie.
Re: Let's be positive (Score:3)
The problem is he's conflating the two facts of who commits more crime and why.
He didn't even talk about why, so how can he be conflating it with anything?
Black people commit more crimes in western nations because of a variety of reasons, the primary being socioeconomic status. Control for that factor across black populations and all of a sudden blackness isn't the thing to look for when predicting criminal behavior.
That's wonderful; even if you had incontrovertible evidence for this and could convince every single person on the earth that "socioeconomic status" was the one and only reason for the difference it would still have absolutely nothing to do with what he was talking about.
However, if you really want to go off topic, consider the fact that your explanation says nothing about why there's such a big difference in the "economic status".
Re: (Score:2)
Re: (Score:2)
The end of Jim Crow was only half a century ago, and of course it didn't magically make all the problems go away overnight.
I was exaggerating, but you really are that ignorant it seems.
Re: (Score:2)
Re: Let's be positive (Score:2)
Racism didn't end with Jim Crow
Yep, this is also why the Japanese American community is so crime ridden and empoverished. Because racism didn't end with the closing of the internment camps.
Re: (Score:2)
When there are no longer directly attributable effects.
Re: (Score:2)
we have had a black president who was elected 2X. there is literally nothing a black man cant do these days.
BR perpetual victimhood does no one any favors
Re: Let's be positive (Score:2)
They were kidnapped, made other people's property, oppressed and systematically discriminated against for centuries.
That's the simple knee-jerk answer, sure. It doesn't explain why other similarly oppressed groups have thrived, nor does it explain why murder and violent crime rates in the black community have increased as opression/discrimination have decreased. Like most simple "answers" it raises more questions than it answers. It's only satisfying to those who prefer not to think too hard.
Re: (Score:2)
It's the start of a much longer explanation, that definitely isn't "black people are dumb/violent".
Re: Let's be positive (Score:2)
Like "white people are racist slave owners"? I guess that is technically longer ...
Re: (Score:3, Informative)
The problem is he's conflating the two facts of who commits more crime and why.
Black people commit more crimes in western nations because of a variety of reasons, the primary being socioeconomic status. Control for that factor across black populations and all of a sudden blackness isn't the thing to look for when predicting criminal behavior.
The problem with your "paint" theory or "blank slate" theory is that there are no successful prosperous black-run areas that are safe pleasant places to live. Including when blacks control all of the political and economic activity. Haiti is a good case in point. It was a productive agrarian economy with things like public sanitation and law enforcement while the French ran it. When the blacks intercepted a shipment of muskets, ammo and powder they overthrew the French and had their own version of the A
Re: (Score:3)
You do realise that 'black' is merely a label for a range of skin colours, races and (at a global level) cultures?
Me, I'd happily live in Morocco. It's fucking awesome.
It's as though Jim Crow served to protect whites from an inherent trait that our ancestors recognized.
Ah, sorry. My bad, I didn't realise I was replying to an idiotic fuckwit.
Re: (Score:3, Interesting)
Hmm, but again, I don't think it has to do with skin colour but with ethnic upbringing, the set of core values, the level of pride and its worth against the worth of other humans life. E.g. Afghanistan is a country of virtually all-white people. Yet, of all the lethal violent crimes among recent migrants in Europe, most of them is done by Afghan young men. Or people from Balkans, or people from most parts of former Soviet Union, so often actors in crimes, even organized crime, also all white.
Black people ei
Re: (Score:2, Insightful)
When I hear such claims I like to use an example that perhaps highlights the problems itself better:
Take a baby and isolate it away, perhaps in a dungeon (or elsewhere horrible) or in the wild with wolves, and wait until it has fully matured.
Now take that person out of that isolation, give them 1 million dollars and toss them out into the real world.
What do you expect will happen to that person?
Yet this is EXACTLY what we did with people and countries. Threw them out, gave them money (if at all) to dampen o
Re: (Score:2)
Same thing was said about East Asians until Japan and south Korea proved it to be bullshit.
I'd also point out that the US was pretty violent and lawless when it was then same age as the African nations you are comparing it to.
In time you will be proven wrong. My bet is that Nigeria is the first to really emerge.
Re: (Score:3)
Including when blacks control all of the political and economic activity.
Where is that?
Haiti is a good case in point.
Haiti [countrystudies.us] has been subject to American influence [aljazeera.com]. America controls much of the economic activity. Try again!
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
The UK police have been using a system of racial profiling that says "if you are black, you are probably in a gang" - however, investigation reveals that the system is mostly wrong.
It is wrong, and 'stop and search' does seem to be disproportionately applied to certain skin colours (although I've been stopped and searched - twice in five minutes). Similarly the knife crime in London is heavily skewed to certain demographics - but as https://www.theguardian.com/co... [theguardian.com] suggests, the primary driver is not race.
Me, I'm distressed that these kids don't feel they have better options. That's a gender issue, not a race one.
Re: (Score:2)
Re: (Score:2)
...If you are looking for a suspect in a city of a million people, and this system flags 10 people, ...
You're quite right, flagging 10 people out of a million would be pretty darn good.
The problem is that the article said that it flagged 2470 people at the UEFA Champions League Final which had an attendance of just over 65k. The system flagged 3.8% of the population as suspect.
So using your example of a city with a million people and assuming as you have that the rates hold no matter the population size, the system would flag 38,000 people, not ten. Of those 38k nearly 35k are completely innocent.
In n
Re: Let's be positive (Score:2)
In no world could I represent those numbers as "pretty darn good".
It's pretty darn good when you consider the fact that human beings couldn't possibly cross-reference even a tiny fraction of those 1 million faces against a database of tens or hubdreds of thousands of known criminals, whereas they can fairly easily check 38,000 specific matches picked up by a computer. I'd go beyond "pretty darn good"; it's downright revolutionary.
Re: (Score:3)
Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.
The thing is, if the false positive rate is that bad, I would argue that a "match" isn't good enough to constitute reasonable suspicion/probable cause for an arrest. If they stopped me for an "intervention", based on a system that bad, I'm either free to go, or under arrest. If they won't let me leave, I'm under arrest, and would consider filing a suit over it (mainly to make horribly inaccurate systems like this less attractive to LE).
Re: (Score:1)
They are clever lads. They can figure something out.
01001001 01100110 00100000 01111001 01101111 01110101 00100000 01100011 01101111 01101110 01110110 01100101 01110010 01110100 01100101 01100100 00100000 01110100 01101000 01101001 01110011 00100000 01110100 01101111 00100000 01110100 01100101 01111000 01110100 00101100 00100000 01110100 01101000 01100101 01101110 00100000 01111001 01101111 01110101 00100000 01100001 01110010 01100101 00100000 01100001 00100000 01100110 01100001 01100111 01100111 01101111 0
Re: Let's be positive (Score:2)
Everyone knows that OJ is just a victim of early facial recognition technology.
Re: (Score:2)
Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.
That is positively bad.
Re: (Score:2)
That is bad, IF the police are in the habit of just shooting suspects. While there are cities in the USA that I would expect to do that sort of thing, I've never heard that the Brits are all that big on "shoot first, question the corpse"...
On the other hand, if all they do is pass the pictures on to a human for follow-up (which follow-up does not include "shoot them, then ask questions"), t
Re: (Score:2)
That is bad, IF the police are in the habit of just shooting suspects. While there are cities in the USA that I would expect to do that sort of thing, I've never heard that the Brits are all that big on "shoot first, question the corpse"...
On the other hand, if all they do is pass the pictures on to a human for follow-up (which follow-up does not include "shoot them, then ask questions"), then it's not that big a deal.
Doood! The whole thread is "let's be positive", so I wrote this is positively bad. You know, like a play on words? He said be positive, so I used it in a sentence. Positively. Wow, tough crowd tonight.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
And what if you did miss your tram? Still not too bad?
And your face is obviously one the system has a problem with, so what if you get stopped again the next day? And the one after?
And each day you miss your tram, they check your id, and let you go home.
No biggy, right?
What if you get a little lippy the 5th time it happens?
Have some proper respect for authority, will you? Because it seems you're standing 0.01 meters too close to the tracks, which is a fineable offense under subsection j paragraph 2 of th
Re: (Score:2)
a heavy-set Caucasian man [..] driving a late model beige Toyota Corolla
Must be an immigrant, that aint happening to the natives.
Intermediate false positive rate (Score:5, Interesting)
despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error
I may have concerns about the civil liberty impact of broad-net surveillance systems in general, but the algorithmic deficiencies of this particular system are portrayed incorrectly in this article. I.e., the front-end of the system (the facial recognition system) has a 92% false positive rate, but together with the post-processing in the back-end, the total system has a false-positive rate of 0%. This is similar to saying that the object detection failure probabilities for a ADAS system need to be viewed in the context of the entire system, and it's the performance of the total system that is significant.
Re: (Score:2)
Yeah let's say it's run in New York City with 8.5 million people. They put out a facial ID profile of a suspect and out 8.5 million people 13 possible suspects are returned. That's a lot better than stopping and questioning every "black guy around 6 feet tall who is wearing wearing shoes in a 2 mile radius".
Re: (Score:2)
Re: (Score:3)
All of them, until they commit another crime (Score:2)
> how many of the 173 people that were arrested as a result of the system does the South Wales Police dept. think might have otherwise been overlooked in the crowds? If that's a significant fraction of those 173 arrests
Based on my experience, I'd estimate that approximately zero would have just been noticed by a cop saying "hey that guy looks like someone who has a warrant". There are a LOT of wanted criminals, far too many to memorize each face. Very few fugitives are caught that way.
They do tend to get
Re: (Score:2)
the total system has a false-positive rate of 0%
A quick fact check on that assertion :
of the 2,470 alerts from the facial recognition system, 2,297 were false positives... the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts ...
saying that despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error.
I think someone is not being totally honest with the facts here, and experience tells me it might be the guy in uniform on a power trip.
Re: (Score:2)
92% of positive results being incorrect is not a 92% false positive rate. Without knowing the total number of images checked, the numbers in the article are worthless for making any judgement.
Re: Intermediate false positive rate (Score:2)
Total attendance was just over 60,000 so that gives you some basis for running your numbers.
Re: (Score:2)
Then that is under 4% positives.
Re: (Score:2)
False positives I mean.
Re: (Score:2)
To phrase this another way, since it seems your point was missed....
450+ people were arrested due to facial recognition.
2470 alerts were generated, and 2297 were false positives. This means 173 alerts were legitimate.
This means 'at least' 277 arrests were based on false positives.
Re: Intermediate false positive rate (Score:2)
No, people are just mixing up numbers. The detection rate refers to natches made during one specific event (a football game) the arrests refer to a much longer period of time. How many of those arrests originated from data gathered at the actual game is not stated.
Re: (Score:2)
Ah, I hadn't caught that.
Thanks for the clarification.
Re: (Score:2)
It actually makes sense (Score:2)
For police work, identifying suspects, false positives only affect the overhead portion - rejecting someone identified. If however it had a false negative, then it would be an issue as it would let people who should be suspects go away free. For the moment, as long as they aren't looking for too many people , false positives just allow them to remind the LEO fearing folk that there is law and order in the land.
What is dangerous is that if the rate does not improve, and you have 10/% of their population doin
Re: (Score:2)
I basically agree, but I will note that you are making one false assumption. You're assuming that the false positives are uniformly distributed over the population.
Re: (Score:2)
Indeed, thanks for catching that. I was also assuming that the criminals are distributed equally between all facial phenotypes
Re: (Score:2)
Well, not precisely. Since we're going into analysis, what's being assumed is that convicted criminals are evenly distributed. There's a lot of evidence that indicates that the laws are unevenly enforced against groups based on phenotype. (Which phenotypes are significant varies.)
So this could be another self-fulfilling prophecy.
Re: (Score:2)
Because of the number of false convictions, it seems unlikely in the extreme that such false positives have led to an erroneous conviction. I'm afraid that conviction for drug charges, with mandatory sentencing, has been particularly problematic in the USA for decades. There is also a strong racial trend towards convicting black men, innocent black men, of drug crimes. I'm afraid this will be exacerbated by facial recognition systems that do not differentiate well among black people's faces.
That's worse than polygraphs (Score:2)
Re: (Score:2)
Re: That's worse than polygraphs (Score:2)
He seems to think that investigating is illegal. I'm not sure why.
Re: If the police randomly stop you (Score:2)
And hold you for questioning, they haven't arrested you. But you still have to be sent to the police station and held in a cell for a day. They still haven't arrested you, they only got you in to question.
That's not how that works. If they are holding you then they have arrested you. They may not have charged you with anything, but they certainly have arrested you.
Typically that doesn't happen unless you refuse to speak with them in the first place. In the vast majority of cases their investigation will consist of:
1. Looking at the match and determining immediately that the computer was wrong.
2. Stopping you and asking you for ID and then determining that the computer was wrong.
3. Questioning you furth
Yellow Alert (Score:1)
"We"? Don't checks and balances typically require outside stakeholders to be directly involved?
Re: Yellow Alert (Score:1)
Re: (Score:1)
Bad maths or fishing (Score:3, Interesting)
2,470 alerts - 2,297 false positives = 173 true positives.
>450 people arrested from "facial recognition efforts".
Either that means there were >277 false arrests due to facial recognition, or they are counting arrests due to "facial recognition efforts" as also including the results of things they found when the searched people based on those false positives.
Since they claim "no one has ever been arrested due to such an error", so this means that both that the number of successful arrests has been inflated to make the system look more useful, and that the system's primary function is to justify illegal searches.
Re:Bad maths or fishing (Score:4, Interesting)
Or (and I'm going out on a limb here) you're getting data from two different sources which haven't been normalised for time, duration, or public relations content.
The real problem to me is, (Score:2)
Just my 2 cents
Simplified algorithm (Score:1)
Here is a proposal for a simplified facial recognition algorithm. It features 0% false negative, at the price of an acceptable false positive rate
bool is_suspect(char *picture, size_t picture_len) { return true; }
And not as robotic! (Score:2)
"That's much better than officers!"
Working as intended (Score:5, Insightful)
Catching criminals is a side effect. The main purpose is to create justification to investigate anyone they want.
Re: Working as intended (Score:2)
Current system: supervisor tells subordinates "We got an anonymous tip that GrumpySteen had sex with a goat. Go investigate him."
New system: supervisor tells subordinates "The computer says GrumpySteen matches the photo of a guy who had sex with a goat. Go investigate him."
What's the difference, exactly? Have you actually thought this through? Or is it just your knee-jerk reaction to scream "uhrmaghurd conspiracy" every time someone develops some new technology?
Re: (Score:2)
Re: (Score:2)
Before you can make that claim you need to show where they said they investigated any of the false positives, as opposed to: "Huh? Nah that's not him! Keep moving."
Re: (Score:2)
Catching criminals is a side effect. The main purpose is to create justification to investigate anyone they want.
No, the main purpose is to catch criminals. The means to that end is creating justification to investigate anyone they want.
There's no reason to assume ill intent here. The police really do just want the best possible tools to do their jobs with maximal effectiveness. It's rarely in the interest of society as a whole to give police everything they want, but that doesn't mean the police are wrong to want it.
Completely hamstringing the police and allowing crime to run unopposed is bad for society. Giving
Re: Working as intended (Score:2)
IMO, having CCTV cameras everywhere is a step too far.
I agree with pretty much the entirety of your very well thought out comment, but I'm curious on what basis you've formed this particular opinion. I myself am rather undecided on how I feel about CCTV surveillance, and to what extent it is or isn't acceptable ... so I would appreciate hearing your thoughts on the matter.
Re: (Score:2)
IMO, having CCTV cameras everywhere is a step too far.
I agree with pretty much the entirety of your very well thought out comment, but I'm curious on what basis you've formed this particular opinion. I myself am rather undecided on how I feel about CCTV surveillance, and to what extent it is or isn't acceptable ... so I would appreciate hearing your thoughts on the matter.
Heh. I wish I had something crisp, clear and well thought out to say on it, but I don't. It just seems like it gives government too much information, too much scope for abuse. Even though there are lots of cameras recording public places in every city, it seems better that the data is fragmented and hard to assemble except upon demonstrable need. Failing that, I'd want to see a very strong anti-abuse infrastructure put in place.
Why don't we compare to current systems? (Score:2)
Another system in current use for doing similar police work is to make public calls for information that might be helpful to a case or broadcast sketches or grainy videos of suspects and ask for the public to call in. What percentage of those calls are false positives? My bet is it is vastly higher than 92%.
Really (Score:2)
I'm more interested in their claim that no one has been arrested due to a false positive. That's nearly impossible to completely avoid without the use of a facial recognition system. Has the UK found a new system that allows them to only arrest guilty people without the need for a costly legal system?
Re: (Score:2)
"Arrest" is specific procedure.
You can "detain" somebody and go through all their stuff without officially "arrest"ing them.
And if you find a little bag of sunshine during their detainment and upgrade it to a full arrest, well, that's how justice works these days, isn't it?
positive = lead (Score:2)
The way investigative policing works is you have numerous leads, and you follow up on them, and most end up asv dead ends, but hopefully some bear fruit.
A false positive is a lead that didn't work out.
Re: (Score:2)
And it's remarkably easy to turn a false positive into a true positive if they happen to be carrying anything illegal when you randomly search them.
The system's so amazing that you can find people you didn't even know you were looking for.
Screening (Score:3)
A percentage, without the context of use, is meaningless.
They might be using them for screening, to focus human evaluation. If so, that means that it is ultimately the cop that makes the decision, not the system. This is how today's AI is meant to be used - as a cognitive aid.
It is fairly common for screening tests in medicine to have high false positive rates. That is OK. They are just meant to narrow down the search space for more expensive/invasive confirmatory tests. Given that the incidence of criminal targets will always be a tiny percent of the corpus, it is very difficult to have tests with high true positive rate. That is quite normal for general tests, in general.
The questions that are relevant are:
1. Are the police able to better solve crime with the aids?
2. Is the test too expensive for the said improvement?
3. What are the rates of negative outcomes (like a wrongful arrest) and..
4. What do we, as a society, consider to be acceptable thresholds?
Context (Score:2)
Using Technology for the Wrong Purpose (Score:2)
This is exactly using technology for something it is completely unsuited.
Facial recognition is useful as second or third-factor authentication of a small and clearly defined user base. Like checking the face of a person wanting to pass a security door whilst the same person is in possession of a RFID badge. Not only do you match against a smallish set of people who "shall pass", but against the very small set of people who may pass with that specific RFID badge, exactly one, that is. And in this case, secur
Re: Using Technology for the Wrong Purpose (Score:2)
This is exactly using technology for something it is completely unsuited.
That's hilarious. To me it seems like they're using it for something it's incredibly well suited to.
It scanned 65,000 people in a public place. Of those it flagged some 2,500 for further assessment. Human officers, who were monitoring the crowd already, then examined the matches and discarded ones which were clearly wrong. The rest were investigated further.
How exactly is that using technology for something it's completely unsuited? Do you honestly think it would be better to have hundreds of cops st
Re: (Score:3)
Re: Using Technology for the Wrong Purpose (Score:2)
Any tool we invent will be abused. You may as well say "I for one don't think fire should be used at all, because anyone who thinks it won't be abused just hasn't been paying attention".
If you're thinking of it in those terms then your opinion on the subject is irrelevant. The question isn't whether it will be abused; the question is whether the benefits outweigh the drawbacks.
ignored (Score:2)
just like a monitoring systems that beeps every few seconds and alerts on eveything which is in some cases important, it will be ignored by its users.
in that case, why keep it running, just turn it off.
Re: (Score:3, Insightful)
I'd rather err on the side of false positives than false negatives (which let them slip away). A minor inconvenience is worth the extra security by far.
Exactly!
A few innocent lives may be lost, but that's a small price to pay for my peace of mind.
Re: When it comes to criminals and esp terrorists (Score:2)
A few innocent lives may be lost, but that's a small price to pay for my peace of mind.
This is, quite literally, the rationale for every form of law (and law enforcement) which has ever existed or will ever exist. Even in the UK, where "police don't carry guns", a few innocent die and many innocents end up rotting in jail. We as a society accept that cost because we understand that no system is perfect but almost any system is better than none. We can try to reduce the number of innocents killed or otherwise injured in the pursuit of justice, but we will never get that number down to zero.
Re: (Score:1)
"It is better that ten guilty persons escape than that one innocent suffer."
- Sir William Blackstone.
"I'm more concerned with bad guys who got out and released than I am with a few that in fact were innocent."
- Dick Cheney.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
how do those boots taste?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
You may wish to revisit and revise your arithmetic.
The UK is not England (Score:2)
There is such a thing as police in the UK but not necessarily UK Police. If there was such a thing, they would be looking to protect all of the citizens of the UK rather than only the English.
I hate to be a pedant but the article focuses on an instance in Wales which is an entirely different country to England, albeit with the same legal system.
Re: They just need to watch out for... (Score:2)
I don't think you understand how population distributions work ...