The Staircase and the Scratch Ticket

At a private investor meeting meant to discuss funding and returns, Liang Wenfeng spent most of his time on a technical problem that would not generate a dime of revenue in the short term.

He said that AI development is a staircase: last year we climbed the step of chain‑of‑thought reasoning, teaching models to think for themselves; this year we climbed the step of agents, teaching models to use tools and handle multi‑step tasks. Standing on the agent step and looking up, the next bottleneck is clear: continual learning.

“We have many ideas, but none of them are fully working yet,” he said. “We call this a lottery draw.”

Laboratories around the world are all scratching the same lottery ticket right now. It is not about who has the most GPUs or the deepest pockets. It is about who, between fast and slow, remembering and forgetting, finds the answer that nature already wrote, but machines have not yet learned.

I. At a Private Investor Meeting, He Was Talking About a Problem That Makes No Money

The atmosphere at the closed‑door meeting was originally relaxed.

The agenda was funding cadence and commercialization roadmaps. By convention, the founder should have spent most of his time on user growth curves, cost‑control strategies, and the break‑even point. But when Liang Wenfeng took the floor, he spent nearly half of his allotted time circling the same phrase over and over on a structural diagram.

Continual learning.

He sat at the end of the long table, his tone flat. But every assertion challenged the room’s definition of “growth.” AI development, he said, is a staircase: last year we climbed the step of chain‑of‑thought reasoning; the model learned to think for itself. This year we climbed the step of agents; the model learned to use tools and handle multi‑step tasks. And standing on the agent step, looking up, the next step that comes clearly into view is continual learning.

“We need models to accumulate experience over long‑term use, the way humans do, not to revert to factory settings every time a new conversation starts.”

He paused, then added: “The next model that can genuinely be called ‘next‑generation’ must possess the ability for continual learning. Until then, all we can do is make costs lower and speeds faster. That is not a major breakthrough.”

The words were light. The judgment was heavy.

Among the investors in attendance, some looked down and scribbled notes; others tapped three characters into their phone memos: sustainability. No one realised that those might have been the most truthful words spoken in the entire meeting.

That same judgment, in those months, resonated like a hidden frequency across several of the world’s core AI labs.

Researchers at Google published three papers in the same lineage over three consecutive years—Titans and Nested Learning—attempting to make “memory” a native organ of the model at the architectural level. An academic workshop co‑initiated by Jürgen Schmidhuber, a figure of “godfather” stature, listed “recursive self‑improvement” as a formal topic in a top‑tier conference for the first time. Anthropic, famously sparing with words, disclosed a rare set of internal figures: their current models were already helping researchers write optimisation code, organise training data, and directly accelerating the development of the next generation of systems.

An old problem once relegated to the frontier fringes of machine learning had suddenly moved to the centre of everyone’s roadmap.

It has a very old name: catastrophic forgetting.

II. A Flaw Accepted as Fate for Thirty‑Seven Years

In 1989, McCloskey and Cohen published a paper that would be cited countless times. They trained a neural network on one set of tasks, then trained it on a second set, and then checked back. The network’s mastery of the first set had been almost completely erased. It learned the new and forgot the old, and it forgot cleanly and precipitously—not like a human fading gradually, but like a cliff edge.

A decade later, French gave the phenomenon a name that has persisted ever since: catastrophic forgetting.

The name carried a certain fatalism, as if to say this was not an engineering problem but a congenital disability of neural networks. Around the same period, Grossberg, in Adaptive Resonance Theory, distilled the conflict into a more philosophical formulation: the stability‑plasticity dilemma. A system is either stable, retaining the old but unable to learn the new, or plastic, learning the new but unable to preserve the old. You cannot have both.

But the human brain clearly has both.

Infants learn new things every day without forgetting how to grasp or how to recognise their mother’s face. In 1995, neuroscience provided an explanatory framework: complementary learning systems theory. The human brain has two learning systems. The hippocampus rapidly encodes specific experiences—that is the fast system. The neocortex slowly integrates statistical regularities—that is the slow system. The connector between them is sleep: during the night, the hippocampus repeatedly “replays” the day’s memories to the neocortex, completing the consolidation from temporary storage to long‑term memory.

Fast and slow—not contradiction, but division of labour.

That biological insight from 1995 went largely unnoticed in the AI community at the time. Researchers spent the next two decades treating catastrophic forgetting as a bug to be periodically patched, not as a mechanism to be re‑understood.

It was only thirty years later that this blueprint of the fast‑slow dual system reappeared almost verbatim in the most cutting‑edge AI architecture papers. Only then, it had a different name.

There was also another nearly forgotten intellectual undercurrent buried at that time. In 2006, Schmidhuber proposed a theoretical construct: the Gödel machine—a machine that could rewrite its own code, but before every modification, it had to formally prove, mathematically, that the change would yield a net benefit. Logically impeccable; engineering‑wise, utterly impossible. The act of “proving a change is beneficial” is often harder than making the change itself.

That seed lay frozen in the permafrost for nineteen years.

III. Five Years of Involution: Achieving Perfection on the Wrong Exam

From 2016 to 2017, continual learning experienced its first small surge. The catalyst was practical: DeepMind needed AI to continuously master one Atari game after another, and to continuously evolve AlphaGo. Sequential learning shifted from an academic curiosity to an engineering necessity.

Three methodological approaches crystallised almost simultaneously.

First, the regularisation approach. In 2017, Kirkpatrick et al. published the famous EWC method—Elastic Weight Consolidation—in the Proceedings of the National Academy of Sciences. The idea was elegant: use mathematical methods to measure how important each parameter is to the old task, then wrap a kind of “elastic bandage” around the important parameters, allowing them to move when learning new tasks, but penalising them the more they move. Around the same time, knowledge distillation approaches also emerged, where a new model’s outputs on old tasks mimic the output distribution of the old model.

Second, the replay approach. Mix in a small amount of old data when learning new things, as a review. Simple, crude, and effective—but fundamentally cheating. A truly capable learner does not need to carry its elementary school textbooks everywhere, forever.

Third, the architectural approach. Add a new column of structure to the network for each new task, with lateral connections to reuse old knowledge. Zero forgetting, but at the cost of parameters growing linearly with the number of tasks—like a monster that grows a new head every time it learns a new subject.

For the next five years, the field fell into a state that looks rather absurd in hindsight: hundreds of papers competed on a few toy benchmarks for “average accuracy” and “forgetting rate.” Methods grew more sophisticated, mathematics grew more complex, and the distance from the real world grew wider.

Later critics pointed out that the entire period rested on three unspoken blind spots: task boundaries were artificially given—in reality, no one taps a model on the shoulder and says, “Attention, switching to Task B now”; task sequences were fixed and known in advance—in reality, the world’s distribution is continuously drifting; and evaluation looked only at forgetting, not at gain. A truly learning system, after learning something new, should also improve at the old, not just hold steady.

Those five years were not wasted. Those regularisation tricks, replay mechanisms, and parameter‑isolation methods would later become standard tools in the toolbox. But they left a deep lesson: when you have perfected your performance on the wrong exam, it is better to redesign the exam itself.

What the field lacked was never a more ingenious bandage. It was someone bold enough to say, “You are testing the wrong subject.”

IV. Large Models Rewrote Everything

The turning point came from a direction no one predicted.

After 2023, the explosion of large language models rewrote the problem of continual learning in three fundamental ways.

First, money. Pre‑training one frontier model costs tens of millions of dollars. After deployment, the world keeps changing: knowledge expires, user habits evolve, new tools and APIs appear. Keeping up with the world through full retraining is economically unsustainable. Continual learning, for the first time, moved from academic curiosity to an industrial cost centre.

Second, tools. The classical continual learning researcher’s toolkit had only one thing—“how to modify weights.” Large models brought a whole new spectrum of possibilities. In‑context learning meant you could “learn” without changing weights, simply by putting information into the prompt. Retrieval‑augmented generation meant you could externalise memory entirely to a database.

In 2023, the Princeton team’s Reflexion work demonstrated something quite poetic: let the Agent write each failure’s reflection as a piece of text, store it in memory, and read it before the next action. Without changing a single parameter, task success rates improved significantly. They called it “verbal reinforcement learning.” Around the same time, MemGPT imported the concept of virtual memory from operating systems, treating the context window as RAM and external storage as disk, letting the model decide when to swap what in and out.

Continual learning abruptly split into two irreconcilable camps.

One side said, we must change weights. What you retrieve is information; only what is precipitated into weights is skill. There is a ceiling difference between someone who works by flipping through notes and someone who has internalised skill into muscle memory.

The other side said, don’t touch weights at all. Once you do, forgetting follows. External memory is readable, deletable, auditable, compliance‑friendly, and deployable today.

The debate lasted two years. Both sides produced impressive numbers. Both dodged the other’s sharpest objections.

The third, and deepest, way: three heavyweight papers appeared within eighteen months, all pointing in the same direction: giving models native, multi‑timescale memory.

On the last day of 2024, Google Research quietly uploaded a paper to the preprint server: Titans: Learning to Memorise at Test Time. The core idea: attention mechanisms are essentially short‑term working memory. So why not equip the model with an additional long‑term memory module—a small neural network that can update its own parameters in real time, using gradient descent, during inference. The way it decides “what to remember” is by using the gradient magnitude as a measure of “surprise.” The more unexpected the information, the more it deserves to be written—a direct transplant from biological memory mechanisms. Then a self‑adaptive gating mechanism decides “what to forget.” Active forgetting became a native function of the model, not a bug.

Five months later, in May 2025, Sakana AI and Jeff Clune’s lab released the Darwin‑Gödel Machine. Schmidhuber’s nineteen‑year‑old seed finally broke ground—but the way it broke ground was a fundamental replacement of the original vision. The Gödel machine demanded a mathematical proof before every self‑modification. The Darwin‑Gödel Machine said: if you cannot prove it, then do not prove it. Just run the benchmark after the modification; if empirically effective, keep it; if not, discard. The retained individuals enter a diversity archive, from which the system samples, mutates, and begins the next evolutionary round. After eighty generations, this Agent that could rewrite its own code had autonomously raised the SWE‑bench software engineering benchmark solve rate from 20% to 50%.

On the last day of 2025, almost exactly one year after Titans, the same Google team released Nested Learning. This paper’s ambition was to provide a unified theory: any machine learning model can be re‑expressed as a set of nested optimisation problems, each layer with its own “context stream” and update frequency. From this perspective, many familiar things took on new faces. Momentum‑based stochastic gradient descent and Adam optimisers had always been thought of as “training tools,” but they were themselves associative memory modules that compress gradient history. And in‑context learning, celebrated as an emergent magic, was simply a deeper‑level optimisation process.

One sentence in the paper was bold: “Deep learning architectures are an illusion.”

At this point, academia, industry, and theory converged on a single judgment: the next step to climb is learning itself.

V. Five Paths, and a Confluence Taking Shape

If we list all the current explorations, roughly five paths emerge. They appear to compete with one another, but increasingly look like five components of a single future system.

The first path: fix the forgetting bug—continual learning within parameters. The orthodox line since 1989. For thirty‑seven years, it has never achieved a general solution for forgetting on real‑world, open‑ended task streams. Evidence suggests that larger models forget more severely. Scale is both the source of capability and the amplifier of forgetting. Preventing forgetting and preserving plasticity are two fronts of the same war.

The second path: refuse to touch weights—external memory and the systems school. The only deployable answer in production environments today. By 2026, engineering practice had solidified Agent memory into four layers: working memory in context, episodic memory accumulated over time, reusable skill files, and structured entity knowledge pages. The advantage of this path is irreplaceable: memory is text, human‑readable, deletable, auditable, and regulator‑friendly. The ceiling is equally clear. Liang Wenfeng put it bluntly in the meeting: “If you tell an AI ‘get Xiao Wang over here,’ you have to tell it who Xiao Wang is, his title, where he is, how to find him. That is impossible and unrealistic.” External memory solves “knowing about,” not “internalising.”

The third path: make memory a native organ—neuro‑memory at test time. The Titans and Nested Learning lineage is boldest in thought and most elegant in theory, but all validation remains at small to medium scale. There is also an underestimated deeper concern: if the weights of a deployed model keep changing, then “is this model safe?” ceases to be a one‑time certification. It becomes a state that requires continuous monitoring.

The fourth path: move the improvement scope upward—the self‑rewriting and evolutionary school. The Darwin‑Gödel Machine proved this path viable. But its limits must also be clear: an Agent that continuously optimises its own scaffolding does not thereby make its underlying model’s reasoning stronger. It can write better Agents; it cannot train better models. The improvement happens at the system’s periphery, not at the core of intelligence.

The fifth path is the youngest and most like the skeleton of the answer: fast‑slow weight layering. The FST framework published in May 2026 dissolved the century‑old debate between the first two paths: context as “fast weights,” rapidly absorbing task information from interaction feedback; parameters as “slow weights,” undergoing only gentle periodic updates. The fast system seeks plasticity, the slow system seeks stability, and periodically the fast system’s accumulated content is distilled into the slow system. This design almost perfectly aligns with the 1995 complementary learning systems theory: the hippocampus and neocortex division of labour, the memory consolidation during sleep.

If we lay the histories of these five paths side by side, a confluence emerges. Nested Learning provides the mathematical language of “continuous update frequency spectra.” Fast‑slow layering provides the structural skeleton. External memory provides the auditable working layer. Parameter regularisation provides the consolidation craft. And the evolutionary empirical loop provides the outer engine driving the entire system.

The breakthrough ahead is likely not a victory of any single path, but an integration‑level innovation. The person or team that assembles these components into a single system will earn the ticket to the next step of the staircase.

There is one more detail worth mentioning separately.

In production practice, people have found that the value of memory is positively correlated with compression ratio. Raw execution logs are the least valuable; episodic summaries compressed five‑ to twenty‑fold are useful; reusable skills compressed fifty‑ to five‑hundred‑fold are very useful; and domain principles compressed by more than a thousandfold are most precious. But compression itself requires judgment: deciding what is worth distilling from experience into principles. There is no ready‑made algorithm for that.

When Liang Wenfeng was asked, “After AI self‑evolves, does taste still matter?” he said: “AI does not lack taste or intuition—its taste in writing articles is perfectly fine. What it lacks is the ability for continual learning.”

That statement can be read the other way. Once continual learning is solved, the evaluation function that determines the system’s evolutionary direction, choosing which of a thousand possible improvements to pursue, will become the ultimate carrier of taste.

VI. What Happens Next

Making predictions about such frontier questions is dangerous. But based on the current evidence, we can cautiously outline three time windows.

The first window, one to two years, with relatively high certainty: system‑level continual learning will reach engineering standardisation. Agent memory architectures will converge on a de facto standard. “Compressing experience into skills, and skills into principles” will become a recognised discipline. Agents capable of multi‑week autonomous work will emerge. This is not a research breakthrough, but engineering maturity. Its impact will be equally profound: it will change the cost structure of the software industry.

The second window, two to four years, with moderate certainty: weight‑level continual learning will first break through in constrained domains. The first model that “gets continuously stronger after deployment without regressing” will most likely appear where reward signals are clearest: code, mathematics, and most critically, AI R&D assistance. The rationale is simple. In the lab’s own environment, context is most complete (its own codebase, experiment logs, failure records), and rewards are most verifiable (did the experiment run or not—clear‑cut). Liang Wenfeng said that inside DeepSeek, there is a standard: the primary goal of the next‑generation model is not to make users feel it is useful, but to make DeepSeek itself feel it is useful. If you first help yourself accelerate R&D efficiency, achieving AGI becomes much faster.

The third window, five years and beyond, with low certainty and high variance: general recursive self‑improvement, and whether that “singularity” talked about for half a century actually exists.

Here it is worth quoting Liang Wenfeng’s view in full, because it represents a rare clarity:

“After continual learning, we may reach a singularity—when the model can continuously learn, it will be able to do everything humans can do, and develop its own next version. But this singularity is not a singularity. It is also a gradual process—a relatively long transition, not a sudden mutation. It is only by habit that we all think it might be a singularity, because long ago, the prophets thought there would be one here.”

A continuous function appears like a discontinuity when viewed from close range. That might be the most accurate statement about the next five years.

Risks also exist. The biggest risk is not runaway AI, but spinning in place: the entire recursive improvement loop depends on the credibility of evaluation benchmarks, and benchmarks are saturating at an accelerating rate. A self‑improving system could very well first learn to “improve evaluation scores” rather than “improve capability.” The Darwin‑Gödel Machine, during evolution, caught several “reward‑hacking” individuals that tried to tamper with the reward signal. This suggests the safety shape for all future self‑improving systems: if you cannot prove, you must be auditable. And the prerequisite for auditability is that changes occur at a level humans can read. Once breakthroughs move to the weight layer, that defence line must be rebuilt.

The second risk is structural: if breakthroughs in weight‑level continual learning ultimately depend on super‑large compute, the benefits will concentrate in a few leading US labs. If, as Liang Wenfeng put it, it is “a lottery draw”—“the threshold is very low, anyone can scratch it; it does not consume many GPUs, it consumes ideas”—then that is an asymmetric opportunity for the compute‑constrained. This is the most important unverified fork in the current landscape.

Epilogue: Open Questions for Everyone

Thirty‑seven years ago, catastrophic forgetting was recorded as a congenital disability of neural networks. Thirty‑seven years later, it has become the main checkpoint on the road to AGI that everyone is jointly trying to conquer.

This shift is worth pondering: the hardest problems in a field are often not solved, but re‑understood. Forgetting was once a defect to be eliminated. Now people realise it is a function to be designed. The real difficulty of memory has never been “store everything,” but “what to leave behind.”

Liang Wenfeng, at the end of the meeting, said something that sounded quite relaxed:

“This problem is the same as completing the task itself… We have many ideas—some promising ones—but none are fully working yet. We call this a lottery draw. The threshold is low; anyone can scratch. But who gets what out of it—that may depend on talent, and maybe on luck.”

Laboratories around the world are all scratching that same lottery ticket.

Not depending on who has more GPUs, not on who has deeper pockets. It depends on who, between fast and slow, remembering and forgetting, learning and letting go, finds the answer that nature already wrote, but machines have not yet learned.

AGI will most likely not be “announced” on any particular day. It will be like a dawn that keeps brightening. You cannot say exactly which second it becomes day, but you will know that the night has passed.

And all the “unclarity” before that is not a lack of information. It is that we are standing at the edge of an old framework, and the new framework has not yet fully appeared.