To Get Smarter, AI Must Learn to Forget

It started with a single support ticket in late 2025. A developer was debugging a production outage—an OAuth timeout that only appeared after two hours of back‑and‑forth with Codex. The model had traced the issue correctly, suggested a fix, and then, mid‑conversation, the context window filled. Codex performed its usual compaction: a neat summary of everything discussed so far. The developer pasted the new error log. Codex looked at it and suggested the exact same fix it had ruled out twenty minutes earlier.

The developer closed the session and never came back.

Inside OpenAI, this wasn't an isolated incident. It was a pattern. And it forced a question that no one had really wanted to ask: What if the way we manage memory is fundamentally wrong?

I. The Default That Stopped Working

For years, the industry had treated context compression as a necessary evil. When a model's window filled up, you called the model itself to summarise the history—turn a thousand lines of dialogue, tool calls, and stack traces into a neat paragraph, then swap that in. It was clean, it was simple, and it worked for demos.

But demos aren't production.

In real, long‑running sessions—the kind that debug distributed systems, refactor legacy codebases, or trace data pipelines across hours—compression did something insidious. It flattened a specific stack trace into "an error occurred." It collapsed a contradictory tool output into "the configuration was adjusted." It made the model forget what it had already disproven, without the model ever knowing it had forgotten.

The failure was silent. The developer only saw a confident, wrong answer delivered at exactly the right moment to waste their time.

There was also a hidden tax. Every compaction required an inference call. For heavy users, those calls ate more than ten percent of their total token budget—a surcharge on the very mechanism designed to save tokens.

II. The Hard Pivot

By early 2026, Codex's core team had seen enough. They didn't tweak the summarisation prompt. They didn't increase the window size. They abandoned the premise entirely.

The new design starts with a brutal insight: the context window is not memory; it is a workspace. Workspaces get cluttered. You don't summarise a cluttered desk; you clear it and bring in fresh paper. The question is: how do you keep working after you've cleared it?

They built three layers.

The Hard Reset.
When the window fills, Codex calls start_new_context_window(). Everything in the current window—every message, every tool output, every back‑and‑forth—is discarded. The system triggers this automatically, or the model can request it proactively. Under the hood, the TokenBudget feature flag reroutes both manual /compact commands and automatic compactions to this hard‑cut path. The old summarisation pipeline is bypassed entirely.

It looks like a step backward. But the engineers knew that the conversation wasn't the task. The task was the thing that kept running, regardless of which window the model happened to be looking at.

The State Checkpoint.
Before the reset, the model writes a structured note. It captures the goal, the decisions made, the progress, the next steps, and—critically—precise references to any historical detail it might need later, tagged with window_id and item_id.

This note is not a summary. It is a checkpoint—active, model‑driven state serialisation, written in the model's own language, placed at the beginning of the next window. The new window reads it and resumes exactly where the old one left off.

The Permanent Archive.
But notes are brittle. They record intent, not evidence. When the model needs to recall a specific error log from two hours ago, the note can't help. That's where History comes in.

History is a lossless, read‑only append‑log of everything that ever happened in the session—every user message, assistant response, tool invocation, and output. It is never loaded into the context window. Instead, the model queries it with a precise index: history.read_item(window_id, item_id). It pulls exactly one fact, one exchange, one stack trace—and leaves the rest archived.

This is pointer‑based memory, externalised and queryable. The context window holds references; the archive holds truth.

The Cycle.
A long session now runs like this:

1. Work in the current window until it fills.

2. The model requests a new window, or the system triggers the reset.

3. Before the cut, the model writes a Note with key references.

4. The system hard‑resets the window.

5. The new window starts with the Note prepended.

6. If the model needs an old detail, it queries History by pointer.

7. Repeat.

The engineers had built something that looked a lot like an operating system: RAM (the context window), registers and cache (Notes), and persistent storage (History).

III. What It Changed

The shift rewired the fundamental failure mode of long‑horizon agents.

Forgetting became active. Under compression, the model could not know what it had lost. It worked from a summary and assumed completeness. Under the new architecture, the model knows that the full history lives externally. It must explicitly decide when to query. Memory is no longer a hidden internal state; it is an auditable operation.

Data stopped disappearing. No detail is ever summarised away. The archive is permanent. At the same time, the model never has to load that entire archive into its limited attention window—it retrieves only what it needs.

Reliability improved, but the success condition moved. The system could now sustain sessions across hundreds of thousands of tokens without degradation—provided the model learned to manage its references well. The bottleneck shifted from context size to retrieval strategy.

IV. The New Headaches

Of course, no architectural shift comes free.

Retrieval quality became critical. The model needed to know when to query, what to query for, and how to interpret what it retrieved. If it failed to query, Notes and History were just dead storage. The system degraded gracefully only when the model exhibited active, disciplined memory management.

Failures became observable but harder to debug. Under compression, a failure looked like a wrong answer. Under hard cuts, a failure looked like a complete omission—the model simply never thought to check the archive. Easier to detect in principle, but requiring entirely new diagnostic tools: trace logs of retrieval calls, audit trails of what the model queried and when.

Engineering complexity went up. Cross‑window IDs, retrieval interfaces, accurate referencing—all of it added overhead. The team had traded a simple, lossy solution for a complex, robust one.

V. The Quiet Shift in the Field

Codex's move was not an isolated experiment. Across the AI engineering community, a quiet realisation was spreading: the arms race of bigger context windows was hitting diminishing returns. Doubling the window size gave you linear scaling in capacity, but tiered memory—fast cache, structured state, persistent archive—gave you algorithmic leverage.

The hard cut is not a downgrade. It is the first practical step toward making long‑horizon agents reliable enough to deploy in production. When models begin to write their own checkpoints, consult their own archives, and request fresh workspaces—as engineers do—they move closer to genuine autonomous operation.

The story that started with a frustrated developer and a ticket about a recurring OAuth bug didn't end with a bigger context window. It ended with a system that finally understood how to forget—so it could keep working.