The Bitter Lesson

For seventy years, general methods that scale with compute have beaten the ones researchers built from what they knew about the problem.

In 2019 I spent four months building an NLP tool by hand. That same year Sutton explained why I was wrong. I called it Süskind, after Patrick, after that realism as obsessive as it is magical, though this isn't the place for that. It was a system to parse sentences from books, feed them into a directed graph, and assist me when I wrote: mid-paragraph, when inspiration failed me or I simply ran out of words, Süskind would suggest turns of phrase drawn from everything I'd fed it. All of it handcrafted and painstakingly thought through. The kind of tool you build when you want to solve a problem by understanding it rather than delegating it. I identified patterns. I wrote rules. I refined them. I added exceptions. I added more. In the end I had something that worked 94% of the time and I was delighted with myself.

That same year Richard Sutton published an essay of about 1,100 words that, without knowing it, described exactly what I'd done, and why I was wrong, and that probably tells the history of AI better than any book written on the subject. Years and years of researchers building ingenious solutions had always lost to whoever simply threw more machine at it. More brute power, less human design. That's the whole lesson, and that's why it's bitter, friends.

Sutton's argument is simple and disturbing in equal measure. For 70 years, in subfield after subfield of artificial intelligence, the same pattern has repeated: researchers develop sophisticated methods that incorporate domain knowledge, human intuition about the structure of the problem written down as careful rules. Those methods work well. For a while. And then a brutally simple general method arrives that knows nothing about the domain and scales with compute. And it wins, by a margin that makes the previous methods look like a category error. Each time, the community spent years (sometimes decades) building systems that incorporated human knowledge, and each time, when enough data and enough compute arrived, general methods built on search and learning, without that knowledge, swept them away. Positional heuristics for chess lost to the massive search of Deep Blue in 1997, and Go followed later. Speech systems built on knowledge of words, phonemes and the vocal tract gave way to statistical models and then to deep learning, and the hand-designed features of computer vision gave way to convolutional networks. Sutton calls it The Bitter Lesson. Bitter because it implies that the most refined intellectual work (the kind you're proudest of, the kind that proves you truly understand the problem) is precisely what time will deprecate.

The idea is counterintuitive in a specific way. Scalable methods are almost never better from the start. In 2012, when AlexNet won ImageNet, there were computer vision systems with decades of engineering behind them that still led on other benchmarks: object detection on PASCAL VOC stayed with hand-engineered pipelines until 20141. The trick is that AlexNet had a property those systems didn't: it could improve predictably if you gave it more data and more compute. The others had reached close to their ceiling. AlexNet hadn't even seen its ceiling yet. The question that matters is which one has returns that scale, more than which one works better today. A method that improves linearly with human effort has a natural limit: human effort. A method that improves with compute has a different limit: how much compute you can get. And the price/performance curve for compute has been dropping nonstop for 70 years (Sutton names it as the ultimate reason: Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation). Süskind was clever. It was also a dead end.

1Girshick et al., Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR 2014, where a convolutional network beat the best VOC 2012 result by more than 30%.

Sutton's essay is from 2019. What he couldn't see yet (or at least not fully) is how The Bitter Lesson was going to combine with something that would change the entire speed of the curve. The historical problem with RL for language had always been the reward signal. If you want to train a model to do something well, you need to know when it's doing it well. For open-ended tasks that means paying people to judge the output, which is slow and expensive and gives a noisy signal. But there are domains where verification is automatic. Where a machine can check the output instantly, at zero marginal cost. Code that executes. Tests that pass or fail. Mathematical proofs that are valid or not. In those domains, the RL loop can run unsupervised and indefinitely. This reads to me as the next chapter of The Bitter Lesson. Sutton said scalable methods win. What we're seeing now is that domains where scale can feed itself (where the system generates its own training signal) behave in a qualitatively different way from the rest. In those domains we haven't seen the ceiling yet.

There's a way of reading The Bitter Lesson that is defeatist: 70 years of intellectual work, and the answer was always more compute, so why bother understanding the problem. I think that reading gets the direction wrong. What Sutton describes leaves room for intelligence, because there's a difference between intelligence applied to building systems and intelligence applied to understanding what kind of systems to build. The two decades of wrong computer vision research produced the datasets and benchmarks that gave AlexNet something to learn from. The domain knowledge became training data. The intellectual work that time deprecates is the kind that tries to substitute for scale. The kind that doesn't (the kind that understands the structure of the problem well enough to turn it into a training signal) has a much longer life. Süskind is still in a repo I haven't touched in years. Some day I'll turn it into what it should have been from the start: a set of labels for a model that learns in two hours what took me four months to approximate by hand. That will also be part of the lesson. And it will be considerably less bitter.