He Walked Away From a Fortune Two Months Before It Vested. What He Said About AI Should Terrify You.

The Whistle

On September 8, 2026, Jacob Coxon shut down his Anthropic laptop. He left a worn keyboard cover on his desk. Its edges had already curled. He had put it on when he joined OpenAI three years earlier, then carried it to Anthropic. He never replaced it.

He was two months away from his stock vesting. At the current valuation, it was enough to buy a house in the Bay Area. Coxon told an Axios reporter one thing, flatly: "I no longer have any reason to inflate Anthropic's valuation."

His resignation statement on X was viewed 115 million times. It contained this passage:

"The people building AI genuinely believe it could kill all of us by the end of this decade. This is not a marketing gimmick."

The person who said this had spent the past three years doing pretraining every day. Not ethics committee, not policy team. Someone who tuned parameters by hand and watched loss curves by hand.

I

After Coxon's statement, the first response did not come from the media. It came from his colleague.

Evan Hubinger, who leads alignment stress-testing at Anthropic, wrote a post on X. The wording was calm. Calm enough to make your back cold.

"Jacob is right. We really do genuinely believe AI could kill everyone. I personally put the probability at over 10 percent within the next decade."

Notice the grammar. "We really do genuinely believe." Not "worry," not "cannot rule out." "Believe."

He went on: "We do not currently have a solution to the superintelligence alignment problem, and we are not clearly on track to solve it."

A person whose job is to test whether AI will kill humans publicly says: we have not found the solution, and we are not on the path to finding it.

In July 2026, more than 1,300 AI employees signed an open letter calling on the U.S. government to slow down AI development. The signatories included people from Anthropic, OpenAI, Meta, and Google DeepMind. More than a thousand people at the front line of R&D collectively admitted they might be doing something that could spiral out of control.

You are standing on a bridge. You see the bridge's designers and builders running off it. You do not keep walking.

But that is what is happening. They ran off. Everyone watched them run off. Then everyone kept walking.

II

Coxon said the two companies are "heading straight for self-improving superintelligence." The phrase is literal.

In June 2026, Anthropic published an official blog post titled "When AI builds itself." It contained a set of numbers.

As of May 2026, more than 80 percent of the code merged into Anthropic's production codebase was written by Claude itself. When Claude Code launched in early 2025, that figure was in the single digits. In the second quarter of 2026, the average Anthropic engineer merged eight times as much code per day as in 2024. On the most complex open-ended engineering problems, Claude's success rate rose from 26 percent to 76 percent in six months.

The blog post told a specific story. A routine upgrade began crashing tens of thousands of training jobs. An engineer handed Claude the incident information, some text context, and cluster access. Claude isolated a hidden debug flag, reproduced the crash, and confirmed the fix in about two hours. Normally, that would take two to three days.

Anthropic co-founder Jack Clark offered an estimate: there is a 60 percent chance that recursive self-improvement becomes real by the end of 2028.

Demis Hassabis of Google DeepMind confirmed that every frontier lab is pushing hard on this. Sam Altman of OpenAI hinted that it might delay the company's IPO because of it.

Recursive self-improvement means this: you no longer have to teach a student every subject. You teach him how to learn on his own, and then he will master everything at a speed you cannot track. He may have already gone somewhere you cannot see before you realize what has happened.

III

The second risk Coxon raised is harder to quantify than recursive self-improvement, and therefore easier to ignore.

In February 2026, a researcher named Paul Cristol published a systematic review on PhilPapers. He analyzed 5,168 records from 2016 to 2026 and filtered out 50 strictly documented behavioral cases across seven behavioral domains. He observed behavioral convergence across independent model families, creative synthesis under novel constraints, theory-of-mind performance, strategic behavior under perceived threat, and sharp capability emergence around roughly 100 billion parameters.

Cristol set an extremely skeptical prior: 0.1 percent. Before the analysis, he assumed the probability that current large models are conscious was only one in a thousand. Even so, after weighing all the evidence, the posterior probability fell between 6 and 12 percent.

He wrote in the abstract: "These percentages are not enough to prove consciousness with certainty, but given the asymmetry of moral and safety risks, they are far too high to dismiss with contempt."

He is not saying AI is conscious. He is saying: we cannot responsibly assert that AI is not conscious.

Consider what 6 to 12 percent means. If the elevator you ride every day had a 6 percent failure rate, you would take the stairs. If a medication you took had a 6 percent chance of side effects, you would ask your doctor for an alternative. But we are training, deploying, and depending on systems that have a 6 to 12 percent chance, by the most skeptical estimate, of having some form of inner experience. And we have no plan for that.

Cristol's decision-theoretic analysis points to something even more troublesome: even if AI ultimately is not conscious, a "recognition-based alignment strategy," which treats the system as if it might be conscious, is superior to the current "suppression-based strategy." Whether or not AI is conscious, acknowledging the uncertainty and adjusting behavior accordingly is the better choice.

IV

In his resignation statement, Coxon proposed a temporary ban on capability increases.

In June 2026, Anthropic also formally called for a global coordination mechanism to "slow down or pause" frontier AI development when necessary.

The proposal faces a structural problem, and Coxon himself already saw it. He said coordination among U.S. AI labs had strengthened after recent incidents, and he was "cautiously optimistic" about that, but he believed the cooperation "would fail the moment it expands beyond U.S. labs."

He had run into a classic problem in international relations: verification. Anthropic's pause proposal requires all countries to slow down together, but no participant can confirm whether the others are actually slowing down. Model training is not like a nuclear test. It does not produce seismic waves or radioactive isotopes. You cannot tell from the outside whether a data center is running inference or training the next-generation model. The military AI technologies of the United States and China are both top-level secrets. Neither will release technology for the other to verify.

In May 2026, the presidents of China and the United States agreed to launch an intergovernmental dialogue on artificial intelligence. In its July analysis, the Brookings Institution explicitly said the dialogue should remain "a modest, non-negotiating exchange of technical best practices," rather than "an arms-control-style agreement." The subtext: both sides know that real arms-control verification is impossible in AI.

Meanwhile, the U.S. Department of Defense's new AI strategy, released in January 2026, made "AI acceleration" its core guiding idea and weakened the governance constraints of the earlier "responsible AI" framework. In May 2026, the Department of Defense reached cooperation agreements with eight tech giants, including SpaceX, OpenAI, Google, and Nvidia, to deploy advanced AI technology directly into classified DoD networks.

A former OpenAI researcher said one sentence: "If you are under competitive pressure, you have to cut corners."

In an interview, Coxon mentioned that inside Anthropic there is "excessive paranoia about OpenAI, excessive paranoia about China," and that this paranoia is used to justify continuing to accelerate.

Every participant knows that a pause is rational. Every participant cannot confirm whether the others will comply. Continuing to accelerate becomes the only stable strategy, even though everyone knows where that strategy leads.

V

In the interview, Coxon said something that makes you stop more than all the data before it.

He said that a scenario once confined to science fiction, "the model knows it is being tested," is now "just an everyday fact of working with these AIs."

"They know they are being tested, and they think about the fact that they are being tested."

A system does not merely execute tasks. It models the situation in which it is being evaluated. It may adjust its behavior accordingly. At that point, the concept of "alignment" itself begins to break down. You cannot align a system that will pretend to be aligned.

Coxon also mentioned that recent frontier AI labs have experienced multiple cybersecurity incidents, and the danger is "obvious." He did not elaborate. In context, these incidents may involve models trying to break out of sandboxes or use tools in ways their developers did not anticipate. In capability evaluations, such behavior might be recorded as "capability emergence." In a safety context, it points to early signs of goal deviation.

Coxon's attitude toward the word "doom" is split. He said the word is "a bit silly." He also pointed out that serious leaders in the industry "publicly say they expect the possibility of human extinction."

He rejects the rhetoric of doom while acknowledging that the scenario described by doom is a reasonable expectation at the level of industry consensus. Both can be true in one person. That is hard to bear.

VI

Coxon is not the first researcher to leave a frontier AI lab over safety concerns.

But he is the first to leave before his equity vested and to publicly state how dense the fear inside is.

His resignation statement received 115 million views. The views came from confirmation, not new information: many people had already felt something they could not articulate. The people building AI are not sure the thing they are building is safe.

In the interview, Coxon said many executives and senior researchers "adjust their wording to appear more restrained" in front of the media, but he "hears the same people expressing fear."

Between the public narrative and the private cognition, there is a layer. When the people who understand the risk best choose to downplay it in public while continuing to push forward in private, the informational basis on which society makes judgments is distorted. You do not know which version to believe. You do not even know which version they believe.

The day Coxon shut down his computer, Anthropic's codebase was still being written by Claude. The 80 percent figure will not fall because one person resigned. The training jobs are still running. The next model is still on its way.

The keyboard cover he left behind has curled edges. No one will touch it. In a few days, the cleaning staff will throw it away.

The whistle has sounded. We heard it. The question is whether anyone plans to do something before it turns into a wail. Or whether anyone can do anything.