I feel like a crazy person every time I try to point out that North America is already there. We've just done it in a less reductionist manner, in which multiple scores are used for different purposes, because you can discriminate better with better targeted data.
The IRS rates you on likelihood of fraud, border patrol rates you on your likelihood of breaking residency law, the FBI rates everyone's chances of being a terrorist, credit agencies rate your worthiness of a loan, insurance companies rate your chances of being a cost, banks rate your likelihood of paying a mortgage, your friends rate your worth and you let them, and if you're in the media biz everything comes down to a numbers game. Your income is used to sort you into pidgeonholes in virtually every aspect of your interaction with every social institution.
Your worth as a human is already determined by a set of numbers; we've just done it so much better than China, because nobody noticed.
As far as I know, none of the agencies you mentioned have individual numerical scores for 330M+ Americans and US residents. The closest thing is credit scores (which the article mentions), which are metered and tracked by private companies under a broad set of federal regulations.
More to the point, your potential employers don't get access to your personal life, beyond what you divulge to them and whatever information they get from public records (sex offender lists, states with public arrest records, &c). They don't know how "good" of a citizen you are, your mental health record, or if you've ever gotten a speeding ticket. China's proposed system opens all of that.
I have no problem with that though. Banks should try to predict the likelihood of people repaying loans. Giving out loans likely to fail is bad. For the economy, for everyone else taking out loans, and for the people loaning money. It's certainly much more accurate and much less biased than the old system of having a human determine your worth. Same with the IRS using prediction to spend less resources and perform fewer audits.
Putting a number on someone isn't inherently evil. The old system had numbers too, they were just in people's heads instead of in a computer. And as I said, they were much less accurate and much more biased.
China is trying to punish people with different political opinions or behaviors. The american system doesn't care what your political opinions are, just that you don't commit tax fraud and pay back your loans on time.
Oh, come on. We have our problems as a society but this is a little much. I have two specific objections.
* The financial crisis was partially a result of mortgages being given to people whose financial data would have suggested that they had no business borrowing so much money. In that case, the opposite of your point is true. This shows that attention to data is beneficial to a mortgage system because it helps avoid questionable loans.
* My friends don't rate my worth, because I don't have friends.
That's a bit too much hyperbole for me (and the top comment!). Sure, the US has its issues (and, if you're not taking direct action to change it...well). But, live outside the US for a while, and you'll see the difference. For example, live in an ex-soviet country. Experience the doorman to your apartment building taking notes on what you're doing, who you're with, etc. Wait for the doorman and every other 'watcher' to compile this data and report it to your boss. Wait for it to come up in your annual evaluation. I think you'll see the distinction.
Should we turn credit into a market for lemons? Just ratchet everyone's access to credit down to what's now granted to those with bad credit? Go back to the days of "how respectable do you look walking into the bank?"
Should the IRS spend its finite resources looking for fraud where it's less likely to be?
Should middle-aged people with clean records in Accords pay the same premiums as people 19-year-olds with DUI convictions in WRXes?
Should the state assign friends/spouses by RNG and enforce that you spend equal time with each?
People are different. They have different personalities, different behaviors, different risk profiles. It makes no sense to ignore these things. In the organization examples you've given, at least, it's much better to use data-driven, evidence-based models kicking out scores than to rely on the prejudices and informal gossip networks of gatekeepers - like we did before we were sophisticated enough for records.
Even where scores aren't appropriate, you're going to allocate your resources - like attention - to people you enjoy over people you don't.
China's system is problematic because it's assessing something we think should not be assessed or considered anywhere - loyalty to the Party. But they were likely already doing that anyway, with informal gossip networks among Party officials. Executed correctly, the "social credit score" system should serve that end more accurately, with less corruption and ultimately more egalitarianism. We just don't think the end should be served.
I think the difference is in the secrecy. In the US, at least in principle, the system is not secret - in China secrecy is the rule - the party is not supposed to be judged from outside.
yeah, the one that worries me most is the FBI one because it probably keys off keywords we use on twitter. What a way to chill free speech. As bad as anything that China might do
I just finished watching the Nosedive (Season 3, episode 1) from Black Mirror on Netflix, it shows a similar system. Unlike other Black Mirror episodes its quite well balanced. I would highly highly recommend watching it.
I watched the first 3 episodes of S3 last night, and as soon as I saw the headline for this story, I thought of Nosedive.
Watching Nosedive, I thought how close to it we were already, it's just that we're measuring people in terms of Followers or Likes. This path started I think when we prioritised Google Analytics numbers over the positive effects you create within your audience: clicks, traffic, unique visitors all seem to count more in 2016 than loyalty and shared values that dominated publishing for the prior 100 years or more.
In the age of "personal brand" it makes sense to me that we're going to end up with "personal metrics" and we're just seeing the tip of the iceberg.
At the end of Nosedive I decided if that was the future of my society, I'd stay being me and suffer the consequences. I'd rather be happy in my own skin that be popular, and if I lived in a society that turned popularity into a measure of personal worth, I'd quite happy to be a dissident within it.
I watched it last night and was struck by how much impact had been lost by Netflix transition. OK they were clearly going for humour, but even with bigger production, and good acting the longer running time diluted it so much. We didn't need the star rating explained every 2 mins for the first 20.
Though I think it was that one that's based on a Charlie Brooker story, rather than written by.
I actually paused after to decide if I was going to bother with the rest at all. 2nd episode (as far as I've got) was much better.
It's Always Sunny In Philadelphia used this concept as a plotline, and managed to treat it with more subtlety. Granted, it's a comedy, so the tone was wildly different.
My favorite aspect of this episode is that the main character's drastic decline in quality and ease of life that coincided with her declining rating are the exact same experiences someone with poor credit or a rap sheet would experience right this second. The rating systems, lists and records we use as a metric for access and worth are draconian as it is. Crowd sourcing it just makes it glaringly obvious with a tight feedback loop.
I just finished watching it myself, after having seen it mentioned in your comment. I found it profoundly heavy-handed, and, unlike pretty much every other episode of the series I've seen so far, I didn't enjoy it at all.
I really disliked this season. It's too sanitized, not anywhere as visceral or cringe-inducing as the previous ones. Hell the first episode actually had a happy ending instead of a deeply depressing one; the endings in Black Mirror always had that appeal in that they always uneeringly hammered in that there's no escaping, there are no happy endings and it can only get worse.
It just felt too "plasticky", that conclusions about the worlds displayed were being forced on me, instead of giving some room for thought.
My concern is that China will use the scoring to make moral judgments about citizens. Yes, we have credit scores and DMV scores and some of us wind up on white lists (PreCheck) or black lists (No-Fly) for air travel.
But determining business risk is different from determining whether someone is a moral or immoral person, and then using that subjectivity to control the population. It's value to the regime is in that subjectivity: If you threaten my political aspirations, I'll score you as immoral, report your transgressions on the nightly news and toss you in prison to be reprogrammed.
The US separates church and state to prevent this.
This is what the DMV does with the point system. The FBI and CIA I think also collect statistics and profile relevant individuals. The difference I guess is that most of these systems are in separate silos. The Chinese government is taking the next logical step and linking it all together. Reminds me of a quote
Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.
As I understand it, the reason the world have become more democratic and liberal for generations was because freedom made a society work better. The open societies out competed the closed ones.
If this model really works for China and other non democracies, so they can keep a liberal economy and still control people hard politically, then:
It really does seem like, with so much distraction and so little necessity, we're nearing the end of the democratic stage of the cycle, where people use freedom chaotically.
No, the world became more US like for generations because the US was a relatively homogenous ethnic population (after genociding the competing ethnic groups) that controls a quarter of the planet's natural resources and has oceans for borders, so it rose to economic and military dominance easily.
Trackers and social media companies try to build similar scoring systems here.
If banks or insurance companies can figure out how credit score changes of Facebook friends predicts you financial status, better abandon them if they screw things up.
"Beer was a leading theorist of cybernetics—a discipline born of midcentury efforts to understand the role of communication in controlling social, biological, and technical systems. Chile’s government had a lot to control: Allende, who took office in November of 1970, had swiftly nationalized the country’s key industries, and he promised “worker participation” in the planning process. Beer’s mission was to deliver a hypermodern information system that would make this possible, and so bring socialism into the computer age."
How incredibly dystopian of you, China. This is also pretty funny considering it is exactly the premise the first episode of the new Black Mirror season.
This reminds me of the "Whuffie" social currency from Cory Doctorow's science fiction novel Down and Out in the Magic Kingdom. A person's current Whuffie is instantly viewable to anyone, as everybody has a brain implant giving them an interface with the Net.
Reddit and YC also expose each action to voting and judging, and sum up users into karma scores. We already live in such a rating based moral system. If the reddit karma would decide a person's worth and social rights, that would be pretty dangerous. People would self censor, be less creative, and in general only act in a way that protects the score. It would turn into a kind of self policing.
True, but these are privately owned communities and if you disagree with their standards you can go join some other community or start your own. A Chinese citizen can't exactly go start their own competitor to the Communist Party.
Well, this is bad because it's the Chinese government.
But companies like Airbnb have to use proxy metrics like number of Facebook friends. If I could be proven to be the trustworthy person I am, then a host wouldn't need to look at all that. If I'm known to have destroyed someone's home, then someone else on Turo knows better than to rent to me.
In practice, it's likely to turn into an eBay style "A++++ best person ever. Let me on the bus brilliantly! 5/5" but the idea isn't terrible on its own.
This is clearly a very bad direction for China in itself. Most people reading this instinctively want a "but", and this is one that I can provide that applies to China and Russia (though Russia is really much freer than China):
Western critiques of China and Russia are directed at the regime, but if the regime collapses, there is no on to criticize, but people still suffer. That is, the Western critique is not fully utilitarian: it criticizes the regime for taking measures to ensure stability, without considering the consequences of the regime losing control.
For reference, about 70 million Chinese died after the collapse of the Qing dynasty (caused in part by Western intervention, by the Opium wars), and Russian life expectancy reduced significantly during the power vacuum between the fall of communism and the rise of Putin.
This gives some background on why both countries support their governments: they want stability and strong central government because they know the alternative.
If Westerners want to adopt a moral and convincing approach to human rights in China and Russia, they need to explain how these things can exist without causing a collapse of law and order that would be worse for the ordinary citizen. This would require reigning in the CIA so that countries could loosen the reigns without fear of a color revolution or Arab spring.
With Trump and Hillary, the community on Chinese websites are increasingly disenchanted with democracy. One high-vote answer in Zhihu, or "the Chinese Quora", claims that this election cycle has been a "great educational experience for Chinese Nationalism and Marxism". You'd be surprised how popular Trump is in China, and how much they support their Government.
"Imagine a world where an X government monitors everything you do, amasses huge amounts of data on almost every interaction you make, and awards you a single score that measures how “trustworthy” you are."
This is wonderful. A fully gamified life. It is so dystopian...
America has the credit rating system. With much hand wringing and energy devoted to increasing or altering the score and so on (companies advertising a way to increase it for you, landlords claiming they'll ruin your credit rating if you ask for a deposit refund ...).
Now imagine if the system was even more corrupt and you could pay someone under the table to alter your score directly. Now imagine that is not just a number for your credit worthiness, but a number for your, well general-life-worthiness. There is no way that is not getting exploited, hacked into or gamed. Maybe that is the ultimate goal, it lets those who control this system, control the population?
Imagine an implied threat of getting -150 points for joining a protest. Or maybe your relative is going to court, you imply to the judge he might get a +50 if he lets your nephew walk. Possibilities are endless, as they say.
We already do this. Everyone has a credit score. Everyone gets a background check before renting or getting a job. Let's not pretend we don't have a rating for everyone yet...
I think the moral highground to judge surveillance and human rights issues has been lost. Pre snowden this kind of thing would cause mass hysteria, massive moral grandstanding and call for sanctions in western populations and media.
Now chastened its more of legalese and tiny triumps based on hope rather than scrutiny.
Just as well the sordid culture of using human rights to score moral points and signal cultural superiority is done and the remaining vestiges of apologism and denial should think seriously about self examination and scrutiny lest the ground shifts even further under their feet.
Coincidently, an article[1] just came down my WeChat pipeline about the population clean-up of Beijing. It's a great, classic propaganda article, full of pictures and statistics about how crowded Beijing is, and lots of "wouldn't it be nice if there were fewer people?" and "look how hard your government has had to work, don't you feel sorry for us?".
Basically they're driving out those who don't have Beijing housing registration. The propaganda is addressed to those who will enjoy the relief after the undesirables are gone. No one speaks to the undesirables.
The one-child policy may be over, but the Chinese government's manipulation of the lives of its citizens is alive and well.
I'm halfway through "Weapons of Math Destruction", and this sort of thing is really, really scary. If the amount of damage done in the US already with big data models, with a much narrower focus, for smaller consequences, and ostensibly "good" purposes is any indication, this will be incredibly harmful.
[+] [-] truffelion|9 years ago|reply
The IRS rates you on likelihood of fraud, border patrol rates you on your likelihood of breaking residency law, the FBI rates everyone's chances of being a terrorist, credit agencies rate your worthiness of a loan, insurance companies rate your chances of being a cost, banks rate your likelihood of paying a mortgage, your friends rate your worth and you let them, and if you're in the media biz everything comes down to a numbers game. Your income is used to sort you into pidgeonholes in virtually every aspect of your interaction with every social institution.
Your worth as a human is already determined by a set of numbers; we've just done it so much better than China, because nobody noticed.
[+] [-] woodruffw|9 years ago|reply
More to the point, your potential employers don't get access to your personal life, beyond what you divulge to them and whatever information they get from public records (sex offender lists, states with public arrest records, &c). They don't know how "good" of a citizen you are, your mental health record, or if you've ever gotten a speeding ticket. China's proposed system opens all of that.
[+] [-] Houshalter|9 years ago|reply
Putting a number on someone isn't inherently evil. The old system had numbers too, they were just in people's heads instead of in a computer. And as I said, they were much less accurate and much more biased.
China is trying to punish people with different political opinions or behaviors. The american system doesn't care what your political opinions are, just that you don't commit tax fraud and pay back your loans on time.
[+] [-] totalZero|9 years ago|reply
* The financial crisis was partially a result of mortgages being given to people whose financial data would have suggested that they had no business borrowing so much money. In that case, the opposite of your point is true. This shows that attention to data is beneficial to a mortgage system because it helps avoid questionable loans.
* My friends don't rate my worth, because I don't have friends.
[+] [-] ap22213|9 years ago|reply
[+] [-] closeparen|9 years ago|reply
Should we turn credit into a market for lemons? Just ratchet everyone's access to credit down to what's now granted to those with bad credit? Go back to the days of "how respectable do you look walking into the bank?"
Should the IRS spend its finite resources looking for fraud where it's less likely to be?
Should middle-aged people with clean records in Accords pay the same premiums as people 19-year-olds with DUI convictions in WRXes?
Should the state assign friends/spouses by RNG and enforce that you spend equal time with each?
People are different. They have different personalities, different behaviors, different risk profiles. It makes no sense to ignore these things. In the organization examples you've given, at least, it's much better to use data-driven, evidence-based models kicking out scores than to rely on the prejudices and informal gossip networks of gatekeepers - like we did before we were sophisticated enough for records.
Even where scores aren't appropriate, you're going to allocate your resources - like attention - to people you enjoy over people you don't.
China's system is problematic because it's assessing something we think should not be assessed or considered anywhere - loyalty to the Party. But they were likely already doing that anyway, with informal gossip networks among Party officials. Executed correctly, the "social credit score" system should serve that end more accurately, with less corruption and ultimately more egalitarianism. We just don't think the end should be served.
[+] [-] zby|9 years ago|reply
[+] [-] partycoder|9 years ago|reply
[+] [-] blazespin|9 years ago|reply
[+] [-] mayukh|9 years ago|reply
[+] [-] colordrops|9 years ago|reply
[+] [-] aub3bhat|9 years ago|reply
[+] [-] PaulRobinson|9 years ago|reply
Watching Nosedive, I thought how close to it we were already, it's just that we're measuring people in terms of Followers or Likes. This path started I think when we prioritised Google Analytics numbers over the positive effects you create within your audience: clicks, traffic, unique visitors all seem to count more in 2016 than loyalty and shared values that dominated publishing for the prior 100 years or more.
In the age of "personal brand" it makes sense to me that we're going to end up with "personal metrics" and we're just seeing the tip of the iceberg.
At the end of Nosedive I decided if that was the future of my society, I'd stay being me and suffer the consequences. I'd rather be happy in my own skin that be popular, and if I lived in a society that turned popularity into a measure of personal worth, I'd quite happy to be a dissident within it.
[+] [-] anexprogrammer|9 years ago|reply
Though I think it was that one that's based on a Charlie Brooker story, rather than written by.
I actually paused after to decide if I was going to bother with the rest at all. 2nd episode (as far as I've got) was much better.
[+] [-] oldmanjay|9 years ago|reply
[+] [-] fucking_tragedy|9 years ago|reply
[+] [-] fvargas|9 years ago|reply
https://en.wikipedia.org/wiki/Psycho-Pass
[+] [-] rosser|9 years ago|reply
[+] [-] lagadu|9 years ago|reply
It just felt too "plasticky", that conclusions about the worlds displayed were being forced on me, instead of giving some room for thought.
[+] [-] fratlas|9 years ago|reply
[+] [-] postscapes1|9 years ago|reply
[+] [-] alextheparrot|9 years ago|reply
[+] [-] sogen|9 years ago|reply
[+] [-] 11thEarlOfMar|9 years ago|reply
But determining business risk is different from determining whether someone is a moral or immoral person, and then using that subjectivity to control the population. It's value to the regime is in that subjectivity: If you threaten my political aspirations, I'll score you as immoral, report your transgressions on the nightly news and toss you in prison to be reprogrammed.
The US separates church and state to prevent this.
[+] [-] dkarapetyan|9 years ago|reply
Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.
[+] [-] cscurmudgeon|9 years ago|reply
> The FBI and CIA I think also collect statistics and profile relevant individuals.
Citation needed (especially that they do it for everyone).
[+] [-] maxerickson|9 years ago|reply
[+] [-] berntb|9 years ago|reply
If this model really works for China and other non democracies, so they can keep a liberal economy and still control people hard politically, then:
1984, here we come. :-(
[+] [-] daodedickinson|9 years ago|reply
[+] [-] hilop|9 years ago|reply
[+] [-] nabla9|9 years ago|reply
If banks or insurance companies can figure out how credit score changes of Facebook friends predicts you financial status, better abandon them if they screw things up.
https://mic.com/articles/123452/facebook-to-let-banks-discri...
[+] [-] walterbell|9 years ago|reply
"Beer was a leading theorist of cybernetics—a discipline born of midcentury efforts to understand the role of communication in controlling social, biological, and technical systems. Chile’s government had a lot to control: Allende, who took office in November of 1970, had swiftly nationalized the country’s key industries, and he promised “worker participation” in the planning process. Beer’s mission was to deliver a hypermodern information system that would make this possible, and so bring socialism into the computer age."
[+] [-] idlewords|9 years ago|reply
[+] [-] gragas|9 years ago|reply
[+] [-] mirimir|9 years ago|reply
[+] [-] cpeterso|9 years ago|reply
https://en.wikipedia.org/wiki/Whuffie
[+] [-] visarga|9 years ago|reply
[+] [-] apatters|9 years ago|reply
[+] [-] nullc|9 years ago|reply
[+] [-] arjie|9 years ago|reply
But companies like Airbnb have to use proxy metrics like number of Facebook friends. If I could be proven to be the trustworthy person I am, then a host wouldn't need to look at all that. If I'm known to have destroyed someone's home, then someone else on Turo knows better than to rent to me.
In practice, it's likely to turn into an eBay style "A++++ best person ever. Let me on the bus brilliantly! 5/5" but the idea isn't terrible on its own.
[+] [-] on_my_position|9 years ago|reply
Western critiques of China and Russia are directed at the regime, but if the regime collapses, there is no on to criticize, but people still suffer. That is, the Western critique is not fully utilitarian: it criticizes the regime for taking measures to ensure stability, without considering the consequences of the regime losing control.
For reference, about 70 million Chinese died after the collapse of the Qing dynasty (caused in part by Western intervention, by the Opium wars), and Russian life expectancy reduced significantly during the power vacuum between the fall of communism and the rise of Putin.
This gives some background on why both countries support their governments: they want stability and strong central government because they know the alternative.
If Westerners want to adopt a moral and convincing approach to human rights in China and Russia, they need to explain how these things can exist without causing a collapse of law and order that would be worse for the ordinary citizen. This would require reigning in the CIA so that countries could loosen the reigns without fear of a color revolution or Arab spring.
[+] [-] kristianov|9 years ago|reply
[+] [-] necessity|9 years ago|reply
[+] [-] pamparosendo|9 years ago|reply
What country is he talking about?
[+] [-] rdtsc|9 years ago|reply
America has the credit rating system. With much hand wringing and energy devoted to increasing or altering the score and so on (companies advertising a way to increase it for you, landlords claiming they'll ruin your credit rating if you ask for a deposit refund ...).
Now imagine if the system was even more corrupt and you could pay someone under the table to alter your score directly. Now imagine that is not just a number for your credit worthiness, but a number for your, well general-life-worthiness. There is no way that is not getting exploited, hacked into or gamed. Maybe that is the ultimate goal, it lets those who control this system, control the population?
Imagine an implied threat of getting -150 points for joining a protest. Or maybe your relative is going to court, you imply to the judge he might get a +50 if he lets your nephew walk. Possibilities are endless, as they say.
[+] [-] mempko|9 years ago|reply
[+] [-] throw2016|9 years ago|reply
Now chastened its more of legalese and tiny triumps based on hope rather than scrutiny.
Just as well the sordid culture of using human rights to score moral points and signal cultural superiority is done and the remaining vestiges of apologism and denial should think seriously about self examination and scrutiny lest the ground shifts even further under their feet.
[+] [-] girzel|9 years ago|reply
Basically they're driving out those who don't have Beijing housing registration. The propaganda is addressed to those who will enjoy the relief after the undesirables are gone. No one speaks to the undesirables.
The one-child policy may be over, but the Chinese government's manipulation of the lives of its citizens is alive and well.
[1]: https://mp.weixin.qq.com/s?__biz=MzA3Mzc3MTc0Mg==&mid=265293...
Edit: Here's another link (sorry, also in Chinese) from June, laying out the bones of the policy to drive people out of the capital. Again, posed as a great policy with no downsides. http://www.china.com.cn/guoqing/2016-06/17/content_38687237....
[+] [-] Tharre|9 years ago|reply
[+] [-] michaelcampbell|9 years ago|reply