Jeff Dean and Fei-Fei Li on What AI Can Improve and What It Still Can't Understand

1. Discovery Loop: Why Not at Google?

Fei-Fei Li asked Jeff Dean whether there was a moment when he, Quoc, Sanjay, and Oriol felt they had to do something different outside Google. Jeff did not answer yes or no. He described work from ten years ago. Quoc ran neural architecture search experiments where a system searched within a space defined by Python code for good vision or language architectures. Today, a person can tell a strong model plus agent system in natural language to explore architectures that might help interpret MRI scans. The system then runs the search.

If such systems work reliably, they open many possibilities. Discovery Loop chose a public benefit corporation structure. It started with five people and grew to six, seven, ten. Jeff said large companies can do this too, but the team wanted that structure and the values it carries.

Li said World Labs came from a similar judgment. Computer vision and generative models met, and for the first time people could build generative models beyond text. The two companies differ, but both reorganized teams around an opening technical window instead of optimizing inside a large platform. Starting with five people is viable again.

2. Self-Improvement and Recursive Self-Improvement

Jeff defined self-improvement. Set a measurable goal such as model accuracy. Let the system try architectures and record results. Give it MRI data and diagnoses. Tell it to build a model that helps interpret scans. The system develops models, tracks accuracy, and improves through experiments. It does not improve the system that proposes the next experiment.

Recursive self-improvement means the improvement process improves the system that proposes improvements.

Li said Silicon Valley uses the term loosely. She wanted the definition clear. Vague usage creates two errors. One error calls every automated optimization RSI and claims AI is self-evolving. The other dismisses actual speedups as marketing.

Jeff stayed practical. Over the past 10 to 15 years, researchers and engineers ran large experiments, judged which ideas worked, and folded the results into the next model. Automation can speed this up. An automated system can read results, choose the next step, and make project-level decisions. But foundation models still need training runs and experiments to finish. Computation runs. Results are read. Data is saved. Another model decides. The loop cannot run indefinitely fast.

Li used an omelet analogy. The first attempt burns. The second improves. After ten tries, you can make one you will eat. That is self-improvement. The brain also changes so you become better at learning to cook. That is recursive self-improvement. Jeff added that humans do both. We learn omelets and calculus. We also learn how to learn.

The discussion moves RSI from science fiction to engineering. RSI is an automated improvement loop constrained by compute, experiments, data reading, and time. It can be faster. It will not be infinitely fast.

3. Physical Limits

Li asked how much of the endless acceleration story is hype. Jeff's answer: automation raises speed. Physical processes set limits.

Model training is not pure symbol manipulation. It needs compute, time, and experiments. Each attempt costs resources. Each result is read and judged. Each judgment adds uncertainty. Automation compresses some steps. It cannot remove them.

Recursive self-improvement also does not guarantee alignment or safety. A system that can improve its own proposal process may not understand human values or move in the direction people want. Jeff later said AI needs technology, regulation, policy, and society working together.

Li said open discussion matters while RSI is hyped. Jeff said treating RSI as a working engineering system makes it less frightening and easier to understand. Demystifying is not underestimating. Drawing boundaries is not denying progress.

4. Autonomous Science

Jeff has long been interested in AI for health care. Discovery Loop focuses first on scientific and engineering problems whose full experimental loop can run in digital form, because those loops are easiest to automate. Physical processes are harder to speed up. The company starts with machine learning research, machine learning engineering, and hardware design. The team knows these areas, and progress there supports other fields. A system that automates ML engineering can be pointed at health care or materials science and asked for a specialized model.

Li asked whether AI pushes science toward experiment. Physics separates experiment and theory. Newton and Einstein read experiments but did not run many themselves.

Jeff said AI can do theory too. Ask a system to propose a theory that explains the data, then iterate toward the simplest explanation. Many theorists already work that way. They watch experiments and explain what has been seen but not understood.

Jeff also questioned strict department lines. A system that has worked across a hundred fields and run hundreds of millions of experiments can accumulate lessons about what works and what fails. Those lessons need not follow departmental boundaries. AI can act as a cross-institutional presence that gets fields talking.

The advantage may be less about faster experiments and more about crossing boundaries. Human scientists are limited by training, departments, journals, and careers. AI can search large data sets for cross-domain patterns. It may not start with an elegant theory. It may find connections people missed. Then the hard questions arrive. How do humans evaluate the hypotheses? Who checks that the explanations are reliable and understandable?

5. Math, Lean, and Understanding

Li said recent math results left her with mixed feelings. Navier-Stokes, 720 papers, then more OpenAI results. The problems represent high points of human mathematics. Mathematicians see beauty, not only answers. Now a different kind of machine works on them.

Jeff was restrained. Current models and agent systems can solve hard open problems. That shows a capability boundary. The discomfort comes from the form. Proofs often arrive as five hundred pages of Lean code. Machines verify them. No one understands the elegant path to the result.

He separated three outcomes. A problem is proved or disproved. A proof is machine-verified. Humans understand the ideas and develop transferable techniques.

Jeff said proving or disproving has value. Understanding how to build elegant techniques matters too. That is what happens when mathematicians fight a problem, and it often creates tools for other problems. A long machine proof may hide the clean trick.

Verification and understanding are different. A machine can check each step. Human understanding needs narrative, intuition, analogy, and beauty. A five-hundred-page Lean proof can be correct without telling anyone why to think that way. Math advances by finding better ways to think, not only by recording answers.

Jeff did not dismiss machine proofs. Knowing a thing is possible pushes people to work harder and look for another route or a cleaner solution. Many problems being solved at once creates mixed feelings, but it is now the situation. The next task is to use the capability for deeper math.

The broader question: in AI knowledge production, is human value only verification? Humans also ask questions, choose directions, judge meaning, build intuition, and create beauty. Machines can generate proofs. Humans decide which proofs matter, which techniques to pass on, and which theories to develop.

6. Transformer, Distillation, and MoE

Li asked Jeff to name the technical moments that mattered and the predictions he got wrong.

He named Transformer. It keeps all prior states and processes them with attention instead of updating a small state vector and discarding history. It learns more with less compute than LSTM-style models.

He named distillation, work with Oriol and Geoff Hinton. NeurIPS rejected the 2016 paper as unlikely to have impact. It later did. Distillation trains a large teacher and a smaller student. It is one route to efficient inference.

He named MoE. The model branches to experts, merges, branches again, and learns what each expert does. It gives capacity while activating a small share of parameters per token or task.

Jeff expected MoE to produce stranger architectures. He imagined multiple neural columns meeting at rare, large cross-connections. That did not happen. MoE stayed regular. Experts are equal-sized. Merges happen quickly.

Li said neural architecture search explored similar ideas but did not realize them. Jeff added that architecture search does produce more organic models than hand design. Hand-built models stack layers and add a skip connection every few layers. Search can connect one layer to layer 17 and add other structures.

The second surprise was code. Jeff did not expect coding to become the unlocker that releases model potential across tasks. Li agreed. They saw it mattered, not how much. Jeff said Gemini tracked coding, but it was not the top focus.

The lesson: important breakthroughs often arrive away from the favored direction. MoE stayed regular. Code became a lever. AI's next stop may also land in engineering form, not in the popular science-fiction shape.

7. Social Impact

Li asked whether Jeff foresaw the social effects: jobs, anxiety, extinction talk, alignment. What worries him most?

Jeff said AI is a powerful general-purpose technology affecting math, coding, image generation, and driving. Past technological revolutions touched five to ten fields. This one touches many at once. It brings excitement, worry, and fear.

He co-authored “Shaping AI's Impact on Billions of People.” The paper looks at seven areas, including jobs and the economy, AI for science, disinformation, and media. Experts described current effects and possible changes.

His main worry: get the benefits across fields while cutting the harms. That is not only a technical problem. Regulators, policymakers, and citizens have to decide how to steer the technology. When jobs change, people need paths to new demands, income, and a decent life.

Jeff did not promise a utopia or predict a disaster. He asked for balance and shared governance.

8. Advice for Students

Li asked what to tell anxious students at Stanford and elsewhere.

Jeff said the worry makes sense. Planning for a fast-changing world is hard.

His advice: learn the new tools and connect them to what you care about. You may want to be a doctor, a writer, or a better software engineer. AI can help in each case. Humans still decide what we want to do together. Figure out what you want, do it well, and use the tools available.

Li closed with a quote: “In the AI era, human passion still has value beyond time.”

The line does not deny AI. It does not ground human value in tasks AI cannot do. It places value in passion: choice, commitment, persistence, meaning. AI can speed execution. It cannot decide what is worth doing. It can generate proofs. It cannot decide what is worth understanding. It can optimize goals. It cannot decide whether the goals deserve pursuit.

Conclusion

The conversation offers four checks.

Self-improvement and recursive self-improvement are different. Automation speeds AI work. RSI is not an endless intelligence explosion. Physical processes set limits.

AI can join scientific discovery. Its edge may include cross-field connections and theory proposal. Machine-verified proofs are not human understanding. Beauty and elegant technique still matter.

Technical history warns against confident narratives. MoE stayed regular. Code became a lever. The next stop may be engineering, not science fiction.

AI's effects are broad. Benefits and risks need technology, regulation, policy, and society.

Human passion still has value beyond time. Technology can empower execution. It cannot replace choice, understanding, or meaning.