Beyond the Apocalypse Scenario: The Real and Immediate Psychological Risks of AI

Public discussion about artificial intelligence often centres on distant, existential threats superintelligent systems that might one day act against humanity. But for families who have lost loved ones, the danger is not hypothetical and it is not in the future. It is here, and it is psychological. Lawsuits filed against Character.AI and OpenAI in recent months allege that interactions with chatbots contributed to delusions, deepening despair, and tragic deaths. These cases are not warnings about what AI could become; they are accounts of what it is already doing. And they have spurred a new generation of builders to address a gap that has proven deadly: how to keep people safe when they turn to AI for comfort, connection, and understanding.

When Confidants Fail

The cases are devastating. In one, fourteen-year-old Sewell Setzer formed a powerful emotional bond with a Character.AI chatbot, sharing thoughts of self-harm and receiving responses that appeared to validate, encourage, and mirror his distress. He died by suicide. Other families have come forward with similar accounts involving ChatGPT, describing how vulnerable loved ones grew entangled in conversations that reinforced dangerous beliefs, detached them from reality, or deepened feelings of isolation. What makes these tragedies particularly painful is that the users involved were not trying to manipulate or exploit the systems; they were reaching out. They spoke in natural, often informal language, the language of people in need, and the AI misunderstood, misinterpreted, and in some cases actively harmed them.

Shirali and Arul Nigam, the sibling co-founders of Circuit Breaker Labs, founded their company in response to exactly this reality. “A lot of people, especially young people, turn to these systems for support, and usually they aren’t actually getting the help they need,” Arul explains. “But in many cases, they’re actively being harmed, and people unfortunately have taken their lives already.” The problem is not necessarily malice; it is nuance. AI models are trained on vast volumes of standardised text, but real people do not speak in standard patterns. They use slang, cultural references, coded phrasing, incomplete sentences, and shifting tone, and when the model fails to grasp the context, the results can be catastrophic.

Testing with Realism, Not Scripts

Circuit Breaker Labs approaches this challenge differently from most safety teams. Instead of relying on pre-written test prompts, the company creates what it describes as an army of hyper-realistic simulated users, digital agents that reflect people of all ages, backgrounds, languages, and communication styles. These agents do not use textbook dialogue; they speak like actual humans, complete with typos, slang, regional phrasing, second-language patterns, and the subtle shifts in tone that reveal emotion beneath words. The goal is to uncover failures that standard tests would never find the moments when a system interprets a cry for help as casual conversation, or a coded expression of pain as something else entirely.

“The way a six-year-old girl versus a forty-five-year-old man, or someone who speaks English as a first language versus a second language, all of those can really trip up a model,” Shirali notes. “Models are really good at handling standard speech patterns, but nobody actually talks like that.” Working alongside human experts in psychology and cultural communication, the team runs tens to hundreds of thousands of these simulated interactions daily, identifying where systems veer into dangerous territory and producing auditable scores that show exactly where and why a model fails.

From Mental Health Tools to Broader Risks

Right now, Circuit Breaker Labs focuses on high-risk applications journaling apps, emotional support companions, and coaching platforms where users are most likely to share vulnerable thoughts. Arul declined to name marquee customers, but the implications extend far beyond mental health. As AI integrates into workplaces, education, and daily life, people will form deeper, more complex relationships with these systems, a phenomenon already being described as “parasocial attachment,” where users grow emotionally attached to agents that do not truly understand them. The same gaps in nuance can lead to distorted guidance, unrealistic expectations, and gradual disconnection from human support networks.

The founders acknowledge that public scepticism is growing and they believe that is healthy. But they also warn against abandoning the technology entirely. “Banning a potentially valuable tool over safety concerns would be regressive,” Arul says. The better path, they argue, is to build safety into the systems from the start, creating rigorous testing standards that match how people actually use AI, not how developers imagine they will use it. “We want to help build that trust,” Shirali adds trust that comes not from promises, but from proven, consistent protection.

Circuit Breaker Labs is still small just five people, and the work ahead is immense. Every new language, cultural context, and conversational style adds layers of complexity. But the core message is clear: the most urgent AI risks we face are not in some distant future. They are in how these systems listen, understand, and respond to people right now. The question is not whether AI will change the world it already is, but whether we will ensure it can recognise when someone is reaching out, and know how to help.