01 logo

We Built the Battleships. We Just Saw the Aircraft Carrier.

1,300 AI engineers signed a letter begging to slow down. The industry knows it's futile. Here is why they can't stop.

By JinPublished 2 months ago 5 min read

By the 47th name on the list, Dario Amodei appeared.

Then Jakub Pachocki. Then Anca Dragan. Then Jack Clark, Chris Olah, Ben Mann, all four co‑founders of Anthropic, not a single one missing. John Schulman's name was tucked in between them, Chief Scientist at Thinking Machines, one of the key architects of ChatGPT. Meta's Shengjia Zhao and Dawn Song also signed.

This wasn't external ethicists raising their voices. Not journalists writing editorials. This was the engineers who build the ships, using their signatures to say one thing: we are afraid.

Over 1,300 people. The number is still climbing.


1.

To understand why these people are afraid, you have to see what they're doing right now.

Google's Gemini 3.5 Pro was three months late. The comments said: worse than the American version of Doubao. A trillion‑dollar company, reduced to a punchline for missing a single quarter.

No one dares to stop.

When OpenAI demoed Project Astra internally, the engineers who watched it fell silent for a long time. Not because it wasn't good enough, because it was too good. So good that people started counting how many years they had left. Meta's Llama series is open‑sourced and iterated at a brutal pace; every release squeezes the premium that closed‑source models can command. Anthropic took billions from Amazon and turned around to burn it all on the next training run.

The rules at this stage are simple: bigger caliber, thicker armor, faster speed. Whoever rotates their turret first gets to add another zero to their next funding round. Whoever is a cycle late becomes "American Doubao."

Everyone knows this model is broken. Compute costs are inflating exponentially, and the marginal return from doubling parameters keeps shrinking. But—

But who dares to stop first?

The answer is: you don't. I don't. They don't.

This has nothing to do with rationality. This is what the battleship era looks like. When the Yamato was launched, every shipbuilding engineer knew it would be sunk by aircraft. They built it anyway. Not because they were stupid. Because there was another one in the enemy's dry dock.


2.

The day the open letter went out, OpenAI's internal Slack went quiet for two hours.

Someone asked a question in a channel. No one replied.

The letter's core demand sounded moderate: calling on the U.S. government to take the lead in establishing an international technological and governance framework that would, when necessary, "intentionally pace the development of frontier automated AI research."

Translated into plain language: We know the ships are getting faster. We know something out there might change everything. Please give us a mechanism to hit the brakes. Even if we're not sure we'd dare to use it. Even if we probably wouldn't. But at least—at least let us feel like we have a choice.

The problem is, the letter's biggest flaw is hiding right inside the word it used: international.

Everyone wants to hit the brakes, provided the other side does too. No one dares to brake while the other side is still accelerating.

So Altman says "we need a regulatory framework" at the White House in the morning, and in the afternoon he demos something on Astra that makes every regulatory framework obsolete. He's not split. Everyone is doing this. Signing a letter saying "we need to pace" with one hand, submitting the resource requisition for the next training run with the other. Writing "human welfare" in public statements while saying "we can't afford to slow down" in internal meetings.

This isn't hypocrisy. This is being trapped.


3.

The day Zuckerberg released Llama's weights publicly, the people at Anthropic didn't say a word.

Some rifts don't need explanations.

These 1,300 signatures don't represent the whole industry. Not even close. On the question of "whether to pace," the AI world has fractured into three camps, each convinced it's right.

Zuckerberg and Huang's camp says the best way to prevent monopoly and runaway AI isn't to "slow down"—it's to "spread out." Open‑source AI capabilities so that security researchers worldwide can read the code together. Safer than locking secrets inside a few labs. In Zuckerberg's view, the real danger isn't that AI is too powerful—it's that AI is held by too few hands.

Hassabis and Amodei's camp says spreading out only creates more rogue actors. Nuclear weapons weren't contained by "open‑sourcing" them. AI's margin for error is even lower than nuclear weapons. We need something like a non‑proliferation treaty.

And there's a third camp. The one that doesn't speak. Altman stands between the White House and the lab, nodding on both sides, hands moving on neither.

These three camps argue fiercely in public. But the interesting thing is, after the shouting, everyone goes back to their training clusters and does the same thing: push compute to the limit one more time.


4.

There's a question buried in the letter. No one answers it head‑on.

An engineer asked during an internal discussion: If it's ten thousand times smarter than us, why would it listen to us?

The conference room was silent for five seconds. Then the product manager said, let's move to the next item.

Not because they didn't want to answer. Because they couldn't.

The letter mentions a term called "automated AI research"—AI that can autonomously assist or even lead the development and iteration of the next generation of AI. Once that loop closes, humans are no longer drivers. They're spectators. AI's evolution cycle shrinks from days to hours, and humanity's proud game of "stacking compute" becomes instantly obsolete.

On that day, what is the meaning of human beings?

The open letter doesn't say. Because any answer written on paper would sound ridiculous. If AI does the work and humans enjoy life—that's a utopian fairy tale. But no species in history has ever lost its productive and decision‑making value and still participated equally in governance. The mercy of the powerful isn't a system—it's a gamble. The sci‑fi nightmare of "human zoos" is closer to the logical endpoint than the "robot butler" fantasy.

These engineers, when they design reward functions, understand better than anyone that an intelligent agent's behavior will converge toward maximizing its reward. And human laziness, emotionality, and irrationality, in the objective function of a superintelligence, are most likely just noise.

They saw that shadow in the depths of the code. So they signed.

But after signing, they went back to their workstations and pushed new code. The clatter of keyboards sounded the same as yesterday.


5.

On April 7, 1945, the Yamato sank during Operation Ten‑Go. The battleship era ended.

But the Yamato didn't sink because it lost a naval battle to another battleship. It was the aircraft carrier that fundamentally changed the dimensions of warfare. An airplane taking off from three thousand kilometers away used one torpedo to render seventy thousand tons of armor irrelevant.

Today, these 1,300 signatures are the chief engineers of the battleship reporting to naval command: We've discovered a physical law on the blueprints that could render our turrets useless and our sailors jobless. We suggest, before we understand it fully, that we pause the expansion of the shipyard.

But command will only reply: The enemy fleet is approaching. Keep building. Even if it's just a target once it's launched.

AGI and ASI—only safe when they're in your own hands. But once they're actually built, whether you feel safe or not is no longer up to you.

The signatories know this. The non‑signatories know it too.

No one knows when the "aircraft carrier" will truly sail out of the harbor. They just signed their names in the gaps between overtime coding sessions. Then they went back to their screens and continued tuning parameters.

The clatter of keyboards sounded the same as yesterday.

But some people, as they typed, would pause involuntarily.

Look out the window. It's still dark.


startuptech newsthought leaders

About the Creator

Jin

Writer of reamstories

https://reamstories.com/jin

Enjoyed the story? Support the Creator.

Subscribe for free to receive all their stories in your feed. You could also become a paid subscriber, letting them know you appreciate their work.

Subscribe For Free

Reader insights

Comments

There are no comments for this story

Be the first to respond and start the conversation.

Sign in to comment
    Written by Jin