The Future of Law Enforcement: The Chinese Social Credit System

Psycho-Pass

I finished Season 1 of Psycho-Pass last week, and it just blew my mind. The story is set in 2112, where surveillance cameras scan the mental state of each passing citizen. The Sibyl System, which the citizens believe to be an A.I. algorithm, performs a cymatic scan and assesses each citizen’s potential of committing a crime, called a crime coefficient. When an individual’s crime coefficient exceeds a specific threshold, he/she is imprisoned and administered therapy until their crime coefficient is lowered down to an acceptable level.

Inspector Akane, the main protagonist who works for the Criminal Investigation Department (CID), the very enforcers of the Sibyl System, is conflicted. She is appalled by the lack of ethics and transparency, but at the same time, has faith in the system because it maintains order and keeps citizens happy. At the end of Season 1, she makes the ultimate decision not to overthrow the Sibyl System, acknowledging Sibyl's role as a peace-keeper and firmly believing that one day a better alternative will be found.

Philosophy of the Sibyl System

The Sibyl System is based on utilitarianism, the idea that a morally right action is one that increases net happiness and well-being. Sibyl ensures the safety of the maximum at the sacrifice of a minority. Although in the anime viewers mainly see high crime coefficient holders banished from society, most citizens enjoy a higher quality of life. In fact, crime is such a rare occurrence that a brutal beating of a woman in the middle of the streets fails to elicit any response from the pedestrians—no one had witnessed a crime before.

An A.I.-run Government?

The authoritarian future dystopia depicted in the Sibyl System sends chills down our spine. How will we integrate technology into law enforcement? Will it look anything like the Sibyl System?

Experts have long been discussing the possibility of an A.I.-run government. Wikipedia defines government by algorithm as “an alternative form of government or social ordering, where the usage of computer algorithms, especially of artificial intelligence and blockchain, is applied to regulations, law enforcement, and generally every aspect of everyday life.” With the advent of A.I. and deep learning, it may only take a few decades for A.I. to be advanced enough to help make decisions in the government.

One form of algorithmic governance is a reputation system. The reputation system is founded on the philosophy that the only reason we don’t all become reckless criminals is because we have our reputation attached to it—in other words, we only act morally because we are being watched. The reputation system takes this idea to the extreme—that if we were being watched constantly, there would be no leeway for people to act immorally. The Sibyl System is an example of a reputation system, with its massive surveillance system, so if anybody has even one thought of doing something unjust, they would be detected and apprehended before getting the chance to do so.

The Chinese Social Credit System: A Reputation System In the Making

When pondering about the future of social ordering, I came across the Chinese Social Credit System, the closest system to a futuristic government by algorithm.

The Chinese Social Credit System is a national reputation system being developed by the Chinese Communist Party (CCP) under CCP General Secretary Xi Jinping’s administration. According to the Chinese government's 2015 Plan for Implementation, the Social Credit System is due to be fully implemented by 2020. Once implemented, the system will “manage the rewards, or punishments, of citizens based on their economic and personal behavior” (Wikipedia). I will explore its mechanisms and the Chinese government's motivations behind it.

Details: What happens when you get blacklisted?

The method is based primarily on rewarding high credit scorers, while punishing low credit scorers. High credit scorers enjoy rewards such as less waiting time at hospitals, discounts at hotels, and a greater likelihood of receiving employment offers. On the other hand, low credit scores are forbidden from purchasing plane and high-speed rail tickets*, their children excluded from school admissions, and their personal information made accessible to the public.

Although different cities have different regulations, behaviors that lower credit ratings are fraudulent financial behavior, jaywalking, and failing to correctly sort personal waste (Wikipedia). Meanwhile, behaviors that boost credit ratings are donating blood, donating to the government, and volunteering (Wikipedia).

Purpose: Why does the government care about promoting trust in the Chinese market?

The stated purpose of the social credit system is to help Chinese people trust each other again. It tackles the problem that Chinese society today is still deficient in trust. Despite the “impressive transformation from extreme poverty to economic giant” (NBC News) in a matter of just 40 years, the legal system in China is underdeveloped. There is no unified credit information system like what we see in many Western societies. Laws alone are inadequate because people default on loans and make counterfeit goods and get away with it.

What are the consequences of the lack of trust? It costs the economy millions of dollars every year. “A company’s reputational loss as a result of financial fraud spreads to the product market in China” (Xin). Because one of the CCP’s major goals is to grow the Chinese economy, there is a massive economic benefit in promoting trust in Chinese markets.

Ethics: Does the Social Credit System invade people’s rights?

“It’s the privacy invasion for me” - The West

This system has been demonized by the Western media for its massive invasion of privacy of its citizens, labeled an “Orwellian nightmare” or a “mass disciplinary machine” by news outlets. Researchers argue that “the credit system will be part of the government's plan to automate their authoritarian rule over the Chinese population” (Wikipedia).

“Yes, but that’s the least of our concerns” - China

Surprisingly, the Social Credit System has gained the approval of many Chinese citizens of all classes and ages. They cannot even trust the products and services offered to them, so they perceive the Social Credit System as something that will discourage fraudulent behavior and promote honest dealings in society. “In a country where consumers must be concerned about toxic baby milk or contaminated strawberries or where internet fraudsters harass hundreds of thousands of people, the social credit system is perceived as a platform for reliable information” (Kotska). Thus, the issue of collection of personal data has receded into the background in China (Kotska).

Conclusion: Will We Live in a Reputation System Anytime Soon?

Probably not. To us outsiders, both the Sibyl System and the Social Credit System seems like an outright invasion of privacy and something we would never agree to. However, we have to remember that the initiative is targeted specifically to solve a problem distinctive to China—the lack of trust in the market. Coupling the CCP's economic goals with its stringent methods, we can understand why it will put forth such an ambitious initiative at the cost of citizens' privacy. Also, even if it works in theory, we don't know if it will be effective in decreasing financial fraud and other crimes. Lucky for us, we will be able to observe a reputation system in play and study its effects.

In the meantime, we should keep an eye out for our own safety. Claims about companies spying on us are not something we should turn a blind eye to (hi there Facebook), and we can expect to be confronted with more digital privacy breaches in the future.

⋅•⋅⊰∙∘☽༓☾∘∙⊱⋅•⋅

P.S. While watching Psycho-Pass and reading articles on the Chinese Social Credit System, there were countless references to the high-tech, authoritarian dystopia described in Orwell's “1984,” so I'll be on my way to read that.

Notes:

*26.82 million air tickets and 5.96 million high-speed rail tickets have been denied to people who were on a blacklist (Wikipedia).