When AI is monitoring humans…

Big Brother is watching you

Decades ago, George Orwell depicted a scenario in his novel 1984: a society surrounded by "telescreens" where everyone was closely monitored.

‘Big Brother is watching you,’ in this famous novel, everyone was cautious in speech and action.Nobody leaks their true feelings,whether joy, anger, or sorrow. We view this novel as an extreme flight of fancy, a plot device belonging strictly to the realm of dystopian fiction. After all, technology wasn't that advanced back then, and installing electronic surveillance in every household would have been prohibitively expensive and not easy to achieve.

Today, however, AI has transformed everything and smartphones are everywhere. The story of 1984 could possibly become a reality…

When Machines Begin to Read Humans

Two years ago,, a Japanese supermarket chain implemented an AI system called "Mr. Smile." It monitors the quality of employees' smiles by evaluating their service attitude based on more than 450 parameters, including greetings, facial expressions, volume, and tone of voice.

Using AI to score employees' smiles sounds more "Black Mirror" than "1984." After all, Big Brother only monitors people's behavior, but now AI also wants to monitor human emotions.

How could  this novel-like setting become reality step by step?

In 1972, American psychologist Paul Ekman published his research on facial expressions, breaking down human facial movements into their smallest constituent units such as how muscles pull, the corners of the mouth turn up, or eyebrows contract and organizing them into the systematic Facial Action Coding System (FACS).

This system comprises 46 basic action units and their combinations that indicates seven basic emotions, including sadness, anger, and happiness. The image below illustrates some of these basic facial action units,such as raising the inner corners of the eyebrows, raising the outer corners, and furrowing the brow. Each of which has a corresponding code.

Since emotions can be decoded, it follows naturally that they can also be learned by machines.

In 1997, Professor Rosalind Picard of the MIT Media Lab published the book Affective Computing. Her research group began breaking down emotions into sets of quantifiable data: speech rate and intonation, minute fluctuations in skin conductance, heart rate variability, breathing patterns, and even changes in body posture.

Building on this foundation, her team developed early emotion recognition systems, for example, prototypes capable of identifying emotional status such as fear, anxiety, happiness, or contentment based on these multimodal signals.

Rosalind Picard’s initial vision was for machines to assist individuals who struggle to interpret emotions—such as children with autism, people with social impairments, or those who find it difficult to recognize shifts in their own emotional states due to stress or depression.

The core issue lay in the reliance on manually designed features: researchers had to pre-define a set of rules. In essence, this approach involved writing a rulebook for machines to recognize emotions

The technological ceiling was not truly shattered until the development of deep learning.

In 2012, deep learning technologies achieved a major breakthrough in image recognition. Meanwhile,  AI gains the ability to cross-reference data, which helps determine, for instance, whether a child is actually slacking off while studying.

In the United States, industries ranging from customer service and finance to banking and healthcare have begun using similar AI systems to infer employees' mental health and work status.

In fact, the customer service industry in China also makes extensive use of AI. It might seem a bit dim-witted—spouting a stream of canned, meaningless pleasantries when you first call—but if you angrily shout "I want to file a complaint" three times in a row, a human representative will quickly pick up. You can try this yourself next time—just make sure your tone is genuinely angry.

And just like that, a fascinating role reversal has taken place: AI, once trained by humans, has turned around to train, monitor, and regulate us. Bosses and parents often assume that tighter oversight inevitably leads to greater efficiency; consequently, whenever a new technology emerges, they are quick to repurpose it into a sophisticated surveillance tool for example, fingerprint scanners, GPS tracking, or AI-powered cameras.

Yet, history has repeatedly shown that wherever there is oppression, there is covert resistance. Moreover, the more advanced the technology, the more ingenious the camouflage becomes.

.

But how to monitor human minds..

In the AI ​​era, this game of wits between regulators and the regulated has become more covert and sophisticated.

When schools monitor "head-up rates" and confiscate phones, students prepare decoy devices or master the art of zoning out while appearing attentive by tracking the teacher with their eyes and maintaining a focused expression, even as their minds wander far away. Similarly, workers devise ways to outsmart AI; a report from the University of Michigan School of Information notes that some employees, when under AI surveillance, deliberately alter their behavior to elicit favorable interpretations from the system.

If all else fails, there is always the option of extending paid breaks for water or bathroom visits. Energy that should be devoted to work is instead squandered on a battle of wits with the boss.

These countermeasures are simple yet effective: you can monitor my posture, but you cannot monitor my thoughts. As depicted in 1984, they never mastered the secret of discerning what another person is truly thinking.

Excessive surveillance often signals an inability to solve the underlying problems. In education, when parents lack the ability to engage their children through the allure of knowledge or the joy of exploration, they resort to a fallback strategy: "Go to your bedroom and finish your homework"

The same applies to the workplace. When bosses lack the insight and capability to drive genuine business growth, product innovation, or market breakthroughs, they shift their focus to what is easiest to measure: processes, attendance, or whether employees are working on  company paper or just replying to emails. It is as if a drop in stock price were caused by employees stealing paper.

Extensive research indicates that surveillance undermines intrinsic motivation in both workplace and classroom

The erosion of motivation caused by such surveillance stems from the fact that it destroys the most precious element of human interaction: trust.

When surveillance is omnipresent, the underlying message is: "I do not trust you to do the right thing on your own." This lack of trust creates a self-fulfilling prophecy: children won't do their homework unless someone is watching, and employees instinctively switch into "slacking off" mode.

Ultimately, AI surveillance does not boost work efficiency, but rather human mastery of the art of pretense.

AI still has a long way to go.