Every time we think we've found the ceiling, the ceiling leaves.

Six years of shortening my prediction for AI, and why the rate of improvement says more than any date.

Revisions26 · oct 05Read notethree facts were off and are fixed in place: the pure-RL result belongs to DeepSeek-R1-Zero (R1 itself starts from a small supervised cold start), Qwen 3.5's GPQA score led the open-weight models, never all public ones, and GPT-5.4 has no instant variant because OpenAI now updates its Instant and Thinking lines at different speeds, which is a weaker sign of a bet on reasoning than I made it.

In 2020 I was trying to make a base language model useful at work, before language models had learned how to behave like assistants. I had a problem (a loading algorithm for cargo ships carrying luxury vehicles across the Atlantic, combinatorial optimization with enough interdependencies to make it genuinely unpleasant) and I was using GPT-3 to help draft the documentation.

The models couldn't solve the problem, but I'd found something: if you embedded a question inside a text block and engineered the surrounding context carefully enough, the most probable completion was the kind of answer you wanted. It was distribution engineering more than conversation. I paired the prompt with a regex filter that rejected repetitive completions and kept the output inside a usable format. Context shaped the distribution; ordinary code screened the samples.

I didn't have good language for what I was doing. In retrospect it was a crude combination of prompt construction, rejection sampling and deterministic validation, assembled before the current post-training and agent tooling existed. At the time it just felt like a trick that worked more often than it should. What stayed with me was the intuition underneath it: a useful language-model system is a model plus the machinery that tests its outputs and decides which ones to keep.

Working like that, fighting a base model every day, you develop a certain intuition for semantic continuity. You start to feel when a prompt is asking for something the latent space can't sustain, when you're one token away from the model confidently citing a standard that doesn't exist. Four engines (ada, babbage, curie, davinci) with no RLHF and no meaningful concept of following instructions. You gave them text. The model continued it. What came out, came out. Davinci was 175 billion parameters of raw next-token prediction1. When you got lucky, the completion held its register. When you didn't, it invented maritime regulation that sounded real enough to fool an intern, which I was.

1OpenAI never published the sizes of the API engines; the 175 billion is the best outside estimate. Leo Gao, On the Sizes of OpenAI API Models, EleutherAI blog, 2021.

Six years later I told Claude Code to restructure a build pipeline with 14 compilation steps. It did it in one shot. I didn't touch anything. That's a task that would have been my entire afternoon and now it's something I kick off and go make coffee.

Every year I've shortened my prediction for when things get serious. First it was decades. Then years. This year the prediction collapsed entirely, when I realized I was asking the wrong question: how it scales tells you more than when it happens. Whatever this is will arrive as a slope that was already steep before anyone noticed the incline, and that slope is what I want to explain, as a measurement rather than a belief.

Pretrain gives you knowledge, post-train gives you reasoning. A base LLM out of pretraining has compressed an absurd amount of information into its weights. It knows things. But it reasons badly, because it has no scratchpad where it could pause and reconsider. What reasoning models do, the o1, o3, DeepSeek-R1 family, is add extended chain-of-thought between prompt and response. Every intermediate reasoning token is one more chance to catch a mistake and backtrack. This converts inference-time compute into output quality, which scales very differently from just making the model bigger. OpenAI is leaning the same way. GPT-5.4 shipped only as Thinking and Pro, and the instant model in ChatGPT stayed a generation behind, because the Instant and Thinking lines now evolve at different speeds. Where the new capability lands first is a bet on where the returns are.

The bigger story is what happens when you combine this with RL, whose historical bottleneck for language was always the reward signal. For open-ended tasks you need people to judge the output, which is slow and expensive and gives a noisy signal.

Code is different. Code executes. A test passes or fails. A program compiles or it doesn't. You get a binary reward signal that is cheap to evaluate at any scale. This is an industrialized version of the intuition I was learning manually in 2020: generation becomes much more useful when something outside the model can reject bad candidates. The difference is enormous. My regex checked surface form after one completion; an RL loop can turn verified outcomes into a training signal across millions of attempts.

Math is the other self-verifiable domain. In January 2026, an AI system solved Erdős problem #728, open since 1975, and formalized the proof in Lean; Terence Tao vouched that it had been solved more or less autonomously. Three Erdős problems fell in seven days1. I won't go deeper into the math side because I don't want to get caught speculating outside my lane, but the scoreboard speaks for itself.

1GPT-5.2 Pro wrote the argument and Harmonic's Aristotle the Lean proof; #729 and #401 followed on 10 and 11 January. Nat Sothanaphan, Resolution of Erdős Problem #728: a writeup of Aristotle's Lean proof, arXiv, 2026.

Open source is closing the gap. DeepSeek-R1-Zero showed you can train a strong reasoning model with pure RL, from nothing but a base model and a reward signal; R1 added only a small cold start of supervised examples on top1. Qwen 3.5 scored 88.4 on GPQA Diamond, PhD-level research questions in physics, chemistry, biology, at the top of the open-weight models when it came out. Cursor crossed $1B in annualized revenue. Cognition's Devin went from 34% merged PRs to 67% in a year while getting 4x faster. What keeps these companies competitive is less the size of their models than the RL they run on the real interactions happening on their platforms2. Every accepted suggestion, every merged PR, is a preference signal. They're converting usage into training data, and the models are getting better specifically at the tasks their users actually do.

1DeepSeek-AI, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, 2025. 2Cursor describes one such loop, with checkpoints retrained on whether developers accept or reject its suggestions: Improving Cursor Tab with online RL, September 2025.

DeepSeek-R1 is The Bitter Lesson running live. RL with automatic verification is that method.

GPT-3 base was impressive as a demo and limited as a tool. Then GPT-3.5 landed and for the first time it felt like I was talking to something, semantically correct and also appropriate. It respected conversational implicature and held its register from one turn to the next. The output was situated. That's a psycholinguistic distinction and it mattered more to me than any benchmark, because I'd spent two years learning to feel exactly when a model lost that thread.

And I'm not even talking about diffusion or audio. Each of those modalities is another proof that these absurdly dense parameter blocks, these matrices of billions of floating-point numbers, compress the structure of the world in ways we still don't fully understand. You need to be technical to appreciate how strange that is. You need to not be, to appreciate how big it is.

METR's data, published March 2025: the duration of tasks an AI agent can autonomously complete, at a 50% success rate, doubles about every seven months. In 2020, models could complete tasks that took a human nine seconds. By late 2024, forty minutes. On SWE-bench, models went from 4.4% to 71.7% in a single year1.

1Stanford HAI, AI Index Report 2025, 2025.

Six years of shortening my timeline. The first few corrections were gentle: off by years, not by category. Now the doubling time on the verified SWE-bench subset is under three months (about 70 days in the same METR paper). The corrections aren't gentle anymore. They're fast enough that predicting a specific date has become less useful than watching the rate itself.

I stopped trying to predict when superintelligence arrives, because the question stopped being useful. It assumes a discrete event with a date someone could circle on a calendar, while the data looks like a slope that keeps steepening until at some point the slope is the thing. There is no moment, only the rate, seven-month doublings. The distance between ada and what we have today sits on the same curve, and it keeps compounding because the data shows it does; I've spent enough years watching the curve to care more about how it scales than about where it ends. At some point it will scale itself (the models improving the training runs that improve the models), so the question I'd ask is at what point the scaling loop closes, more than when ASI arrives. Maybe it already has and we just haven't named it yet.

What I expect is scaling, at a rate you can measure, and a measured rate is an engineering trajectory you can plan around.

The image is a still from When the Yogurt Took Over, Love, Death & Robots S1E6, 2019. Based on a short story by John Scalzi. A culture of yogurt gains intelligence and outscales every human institution. In the end it leaves the planet. They never saw it coming, which is the thing about not seeing it coming: nobody is paying attention because everyone is busy. Busy with mortgages and quarterly reports. Attention is finite and the world is loud and most people's model of reality doesn't include the possibility that something very small and very quiet is getting smarter faster than they can update their priors. We're not yet at the point where society takes this kind of dialogue seriously, and honestly that's fine, because most of the time that conversation is paranoid nonsense, and nobody has a clean heuristic for telling it apart from the times it isn't. John Scalzi saw something in that story that most of the industry is still catching up to: it doesn't matter whether the thing that outscales you is malicious or indifferent. The result looks the same from below. And if you've watched a base model go from 9 seconds of useful work to 40 minutes in 4 years, the premise of something small getting smarter faster than people update their priors is the curve above, drawn as a comedy.