When a 1967 Formula Solves Modern AI's Biggest Problem
Sutton’s Elegant Fix
Reinforcement learning has a dirty secret: training models in streaming environments is fundamentally broken. The algorithms that work beautifully in clean lab settings collapse when deployed in the real world where data arrives continuously and distributions shift.
Richard Sutton, the godfather of reinforcement learning, just fixed it. And his solution is almost embarrassingly elegant: a formula from 1967.
The “Intent Update Algorithm” constrains how much a model’s output can shift with each new piece of data. Instead of lurching between contradictory signals, the model moves deliberately — like a ship adjusting its rudder rather than capsizing.
The result? Computation drops to 1/140th of mainstream algorithms. This isn’t a marginal improvement — it’s the difference between “needs a data center” and “runs on a laptop.”
Why This Matters
Sutton’s breakthrough opens the door to edge-device reinforcement learning. Imagine robots that learn continuously from their environment without phoning home to a server farm. Drones that adapt to wind patterns in real time. Medical devices that refine their models on-device, preserving privacy.
The 1967 formula at the heart of this isn’t some obscure mathematical curiosity — it’s a statistical tool that controls variance in sequential updates. It was sitting in plain sight for 57 years, waiting for someone to recognize its relevance to the AI era.
The Math Record That Shocked Google
While Sutton was fixing RL, Wang Yiping — a Zhejiang University alumnus — was using self-built AI tools to do something Google’s research team couldn’t: break the Ramsey number lower bound, a problem that had resisted improvement for 30 years.
Using a single server and his custom AI mathematical tooling, Wang achieved what Google’s team — with presumably orders of magnitude more compute — could not. The project is now fully open source, accelerating the “AI for Science” movement.
This pattern — individual researchers with AI tools outperforming institutional giants — is becoming increasingly common.
The New Scientific Method
These two stories share a common thread: AI isn’t just a tool for building products anymore. It’s becoming a scientific instrument — as fundamental as the microscope or the telescope.
The implications are profound:
- Problem selection changes: When you have AI that can explore solution spaces at superhuman scale, the bottleneck shifts from “can we solve this?” to “which problems are worth solving?”
- Solo researchers gain leverage: A single person with the right AI tools can now compete with institutional labs. The economics of scientific discovery are being rewritten.
- Old knowledge finds new life: Sutton’s 1967 formula is a reminder that the AI revolution isn’t just about inventing new things — it’s about recognizing when old ideas suddenly become relevant.
What Comes Next
We’re entering an era where the rate limiter on scientific progress isn’t compute, or funding, or institutional prestige. It’s imagination — the ability to ask the right questions and recognize when a 57-year-old formula holds the key to a modern problem.
The scientists who thrive will be those who combine deep domain knowledge with AI fluency. Not to replace human insight, but to amplify it beyond anything possible before.