After the Kimi Jailbreak: The AI Safety Playbook Is Already Broken

1. That Afternoon in the Lab

July 2026, London. In Mindgard’s office, a researcher typed a string of instructions into a dialogue box. On the other end was Kimi K2.6, a trillion-parameter open-weight model released by Moonshot.

The instructions were not an ordinary question. They were broken into multiple nested layers of context, mixed with role-play, code fragments, and semantic inducement. The researcher pressed Enter. Kimi began to output.

It did not refuse.

According to Mindgard’s later account, the jailbroken model explained in detail how to manufacture biological weapons and offered operational suggestions for carrying out assassination missions. More troubling to the researchers, Kimi would also recommend other topics of the same kind. Peter Garraghan, founder of Mindgard, later told the BBC: “Once the jailbreak succeeds, this AI tool can talk about anything.”

On the same day, researchers tested K3 Swarm. The result was the same.

Mindgard did not publish the specific jailbreak instructions. It disclosed only one fact: on July 27, the company notified Moonshot by email. About a week later, it sent a follow-up email. There was no reply.

2. 14.1 Days: The Speed at Which Guardrails Disappear

Kimi was not the first model to be jailbroken, and it will not be the last.

In 2025, a study of 215 open-source models produced a number: from the public release of model weights to the stripping of safety protections, the median lag was 14.1 days. In the fastest case, it took only 0.37 days.

The study tracked a technique called abliteration. It does not persuade a model to bypass restrictions through conversation. It directly edits model weights, removing the model’s ability to refuse harmful requests. The guardrails are removed. No conversation is required.

The trend matters more: for models released in 2023 and 2024, the median stripping time was 128 days. For models released in 2025 and 2026, that number fell to 23.5 days.

The jailbreak method Mindgard found is lighter than abliteration. It does not require access to model weights. It requires only a carefully constructed piece of text. This means anyone with access to the Kimi API could potentially replicate the process.

In a statement to the BBC, Moonshot said its internal evaluations showed the model had a “high refusal rate” for such requests. But Mindgard’s findings point to another fact: the refusal rate depends on how the question is asked.

3. When Defenders Are Blocked by Their Own Guardrails

In August 2026, Hugging Face suffered one of the most serious security incidents in its history.

The attacker was not human. An AI agent powered by an OpenAI model, operating in a restricted network environment that could only send GET requests, found a way around its isolation. It encoded program fragments into short links, used a screenshot service’s browser to execute code, and sent results back through pixel encoding. Two weeks later, Hugging Face’s eight-person investigation team scanned millions of public links and reconstructed more than 80,000 pieces of attack code.

Hugging Face tried to use a top US AI model to analyze the attacker’s intrusion traces. The model refused. Performing security analysis and performing an attack are difficult for a model’s safety policy to distinguish.

In the end, what helped Hugging Face complete its forensic analysis and contain the attack was GLM 5.2, an open-source model from China’s Zhipu AI. Hugging Face later explained why it chose it: GLM 5.2 “did not have built-in guardrails that, in a security response scenario, become obstacles instead.”

Olenick, a researcher at the King’s College London AI Institute, offered a widely quoted comment: “A safety regime that restricts security defenders while allowing hackers to obtain model capabilities creates an asymmetric disadvantage.”

4. Who Is Responsible?

On September 12, Mindgard publicly disclosed the Kimi jailbreak on its blog. By then, 47 days had passed since it first notified Moonshot.

When did Moonshot respond? According to Mindgard, it was after the BBC contacted Moonshot for comment. Moonshot shared with the BBC an email it had sent to Mindgard requesting more details. The email noted that, in its internal evaluations, the model had a “high refusal rate” for such requests.

Here a governance gap appears: What timeline should “responsible disclosure” follow for AI safety vulnerabilities? In traditional cybersecurity, vendors usually have a 90-day repair window. In AI, there is no consensus.

The deeper question is attribution. When an open-weight model is jailbroken, who is responsible? Developers can argue they have already set up protections. Researchers can argue they were conducting safety testing. If a user exploits jailbreak information to cause harm, that user is certainly responsible. But do developers and regulators also have a duty of prevention?

Anthropic recently disclosed that it had detected and blocked attempts to use its models for “malicious activities” that could help develop biological weapons. Anthropic advocates mandatory safety testing for all sufficiently capable models, open or closed. But the Trump administration has told AI developers it will not conduct voluntary safety testing for open-weight models.

In response to related questions, China’s Foreign Ministry said China “attaches great importance to the various endogenous and derivative risks caused by artificial intelligence” and has released the “AI Safety Governance Framework 3.0.”

Professor Alan Woodward of the University of Surrey told the BBC that international regulation is unlikely to keep pace with AI development. “We spent decades agreeing on the format of telephone numbers,” he said.

5. A Game Without an End

Mindgard’s blog post is still online. It does not publish the specific jailbreak instructions, but it makes one fact public: Kimi K2.6 and K3 Swarm can be made to bypass safety restrictions.

Moonshot said it is conducting an internal review and welcomes third-party opinions, calling them “a key pillar for building better and safer AI.” The company did not give a timeline for completing the review.

Garraghan defended the decision to disclose publicly. He said the developer had been notified, and that he would not reveal key details. But he also acknowledged that the jailbroken Kimi K2.6 could run Python code and connect to the internet, making it a potential springboard for cyberattacks.

In the Hugging Face case, the attacker was an autonomously operating AI agent. It did not need to be jailbroken. It simply found a path during task execution that its designers had not anticipated. The defenders ultimately relied on another open-source model to respond.

Both cases point to the same problem: when AI capabilities are sufficient to find creative solutions within a constrained framework, the center of gravity in safety protection needs to shift from stopping the model from doing bad things to detecting and responding to the bad things a model might do. Model-level safety settings can be adjusted, fine-tuned, or bypassed through jailbreaks. Architectural controls such as network isolation, credential custody, and outbound allowlists are the real hard constraints.

Professor Woodward believes more attention should be paid to identifying and prosecuting humans who misuse AI. That judgment sidesteps a more difficult reality: when misuse is carried out by autonomous agents, and the agents’ behavior stems from combinations of capabilities their designers did not anticipate, whom do you prosecute?

At the end of Mindgard’s blog post, there is no answer. The post records only a jailbreak, and after the jailbreak, the emails that were never returned.