Here's an uncomfortable way to describe the most celebrated AI systems of our time: they are brilliant fossils. A large language model learns voraciously — during training. Then it ships, its weights freeze, and from that moment on it never learns another thing from the millions of interactions it has with the world. It can be wrong the same way a billion times and be none the wiser for it.
We've normalized this so completely that it takes effort to notice how strange it is. No intelligence we've ever encountered in nature works this way. And I've become convinced that this — not reasoning, not context length, not multimodality — is the deepest missing piece in the current generation of AI.
"But what about RLHF?"
The standard objection is that today's models <em>do</em> learn from feedback — there's literally "reinforcement learning" in RLHF. Look closer at where that learning happens, though. The fine-tuning phase where humans rate outputs and the model adjusts happens <em>in the lab, before deployment</em>. It's part of manufacturing, not part of life. (The lineage of the idea is lovely, incidentally — it traces back to research on agents learning directly from humans saying "good job" and "bad job," years before it became a pillar of LLM training.)
The crux of reinforcement learning — the actual idea, the one that recently earned its pioneers the Turing Award — is something more radical: an agent <em>learns from its own experience</em>. It tries an action, sees what happens, compares the outcome against its expectations, and changes itself accordingly. Continuously. After deployment. In the world. That closed loop — act, observe, update — is precisely what we amputate the day a model ships. What remains is pattern recognition of extraordinary breadth, permanently anchored to the moment its training data ended.
What learning from experience actually looks like
Because this can sound abstract, let me make it concrete with a classic line of multi-agent research I love. Take a simple soccer drill — keepaway — three agents trying to keep a ball from a defender in a small space, deciding continuously when to hold and when to pass. Start the agents deciding <em>randomly</em>. Then let them play, alone, learning purely from what works and what fails. Within about a day of experience, they hold the ball roughly two and a half times longer than random. Nobody scripted a strategy. Nobody labeled anything. The team taught itself to be a team, from experience.
The same paradigm — learning partly in simulation and partly in the physical world, with techniques to keep the simulator honest against reality — has produced legged robots that taught themselves to walk faster than any hand-engineered gait for the same hardware. And at the far end of the spectrum, learning from millions of trials produced a racing agent that outdrove world-champion human players in a high-fidelity racing simulator — a landmark precisely because it was a <em>real-time control</em> task with physics, not a turn-based board game, and it graced the cover of one of science's most prestigious journals for exactly that reason.
None of these systems was smart on day one. All of them were smart on day N — because N mattered. That's the property our deployed language models don't have: for them, every day is day one.
The timescale nobody has cracked
Within the learning-from-experience world there's a further distinction I find clarifying: <em>when</em> is the learning allowed to happen? The successes above are slow-loop learning — hours or days of accumulated experience, before the "real game" begins. The much harder frontier is the fast loop: adapting <em>during</em> the encounter. Noticing, mid-game, that this particular opponent always favors one side of the field, and reshaping your strategy on the fly. Humans do this constantly and casually. Autonomous systems, for the most part, still can't — the experimental demonstrations are narrow, and the general capability doesn't exist yet.
Map that onto today's AI landscape and the gap is glaring. An assistant that has helped a specific team for six months should be <em>different</em> — measurably better adapted to that team — than it was on day one, in its actual competence and not merely in whatever transcript got stuffed into its context window. Retrieval and long contexts are memory aids; they are notes taped to the fossil. They are not the fossil learning.
The confluence
Here's why I'm writing this now rather than as a lament. The two traditions — massive pretrained models and experience-driven reinforcement learning — grew up separately, and they are visibly converging. On one side: models with staggering pattern-recognition over language and vision, increasingly wired into robots and agents that <em>act</em>. On the other: a fifty-year-old toolkit for exactly the problem those agents now face — trying things, observing outcomes, and updating internal state after deployment, not just before it.
The interesting engineering question of the next few years, as I see it, is what that marriage looks like in practice. It will have to answer questions the frozen paradigm never faced: how to let a deployed system change without letting it drift into failure; how to accumulate skills over a lifetime rather than master a single task and stop — the continual-learning problem, learning as an ongoing stream rather than an event; how to audit something that is, by design, no longer identical to what you tested. These are hard, sobering problems. Freezing the weights was never just a limitation — it was also a safety rail, and removing it responsibly is a research program in itself.
But the destination seems clear to me. We call these systems "learning machines," and for one glorious phase of their existence they are. Then we embalm them and sell tickets. The next real leap won't be a bigger fossil. It will be the machine that's still learning on the day you use it — better this week than last week, because <em>it was there, and it noticed</em>.
