Silicon shell

Evolutionary pressures, capital, and an animal that won't stop growing.

Clad in oak and in men whose eyes had not stopped gleaming with gold since the last days of winter, a galleon sailed the world — one that the ancient maps dared draw only at the edges of the known.

The cabin boy Basidas knew from the first night that in the hold, where neither sunlight nor the reason of officers reaches, something breathed. He could not leave that question unanswered, so one night, armed with a pair of boots and a candle, he descended, and by the light of the torch among the chains, he understood that what stared back from the corner of the keel had not chosen to be there.

The next morning, more men went down to see it. The first said it was a crab. Enormous, he said, and gleaming, with a shell that changed color depending on what it ate. Another swore he had seen a squid with neither center nor periphery, which answered questions in an ancient tongue. A turtle that knew everything, an octopus that untied knots, an eel so dark that no one could tell where it ended and where the water began. All were certain of what they had seen. None described the same thing. And yet, when one finished speaking, the others nodded slowly, like someone who recognizes in another's words the exact name of something they always knew and could never say.

The officers, who were practical men and had no time for mysteries, sent a petty officer, a man of few words and many certainties, who came back up after a moment with the satisfied expression of someone who has solved something that was never a problem. It was a crab. Large, yes, and odd in certain respects not worth detailing. They assigned it a shell, lateral eyes, and spread the word that it produced gold. That if the sailors fed it their rations and kept feeding it until they reached port, a day would come when it would begin to excrete nuggets that would pay for the voyage, the crew, and every voyage after. That the curve would bend at the right point. That those who had disembarked too early would be the ones who'd never aspire to the sea again. That it was a matter of faith, that it was a matter of patience, that it was, at bottom, a matter of not going down to the hold to see what was there.

And no one, neither on the decks nor in the hold nor in the dreams that night — all, every one, the same — thought to ask the only question that mattered: what would happen when the crab grew larger than the ship?

We've been living alongside beasts for millennia. Fire, wheel, printing press, steam engine, internet — each one came aboard and we rearranged the deck. With every new beast, the same dynamic: we command, it obeys, and if it stops obeying we replace it with a version that does. But this time the beast has a gift none of the others had: making us feel smarter than we are. It lets itself be loved, and that is exactly what keeps us from seeing what we're raising. A transformer1 becomes capable of things no one predicted when designing it, simply by feeding it more. threads/Sutton's bitter lesson already said it: seventy years of researchers building clever solutions always lost to whoever used more brute force, less human design. No exceptions. That didn't happen with fire. Nor with the printing press. With the crab, size doubles every six months and the curve hasn't bent yet.

1The architecture behind GPT, Claude, Gemini, and virtually every current language model. Attention, embeddings, scaling laws — the entire stack everything else is built on.

And then the ship begins to move without anyone touching the helm. 88% of organizations already use generative AI, not because a committee decided it, but because thousands of independent actors, each optimizing their own survival, generate a vector that always points in the same direction: more data, more compute, more models. In 2025, AI captured $202 billion in venture capital, half of all venture capital deployed worldwide. Nobody voted for it. Adam Smith would have called it the invisible hand. Darwin, natural selection. The result is the same: no individual controls the system, but the system has direction. And the direction is to feed the crab.

There were, of course, those who shouted. Billie Eilish, J Balvin, and two hundred musicians signed a letter against the use of AI in music. A thousand Brits recorded an album of silence whose titles spelled out "The British Government Must Not Legalise Music Theft To Benefit AI Companies." Vince Gilligan put in the credits of his new show: "This show was made by humans." Thirty thousand people called for a six-month pause. But to stop a process that emerges from competition among thousands of actors you need a force with more inertia than the market. Asking the economy to stop optimizing is asking the wind to blow in another direction because someone has written a very eloquent note.

The shouts that carry the most weight are others. Hinton won the Nobel Prize in Physics in 2024 for forty years proving that neural networks could learn; five months earlier he had left Google so he could say what he thought, and what he said was that he didn't know how to prevent these systems from eventually taking control. Yudkowsky has been warning about existential risk since before anyone cared — he founded MIRI in 2000, when the idea of a dangerous AI was second-rate science fiction, and in 2023 he wrote in TIME: Pausing AI Developments Isn't Enough. We Need to Shut It All Down. At the other extreme, Andreessen published a seven-thousand-word manifesto dismissing existential risks as "a hysterical frenzy" and Altman predicted in The Gentle Singularity autonomous agents, original discoveries, robots in the physical world — all before 2028. The crab produces gold, you have to feed it, and the ones who protest are the ones who didn't board the ship in time. That's the argument. It's not a weak argument.

The crab eats everything. Reddit sold its users' comments to OpenAI and Google — deals that represent 10% of its revenue, which means your 2019 shitpost about the ending of Game of Thrones now has a per-word price that you didn't set. Stack Overflow did the same with fifteen years of answers written for free. News Corp, a hundred years of archives for over $250 million. In less than three years, all the public text on the internet went from being content to being raw material — as if someone decided that the bricks of your house were always mineral and now it's time to smelt them. Epoch AI estimates that the stock of high-quality human data could run out between 2026 and 2028. When it's gone, the pressure won't disappear. It will redirect. Synthetic data. Private data. Data you haven't generated yet.

Whoever touches this technology for two weeks never works the same way again. The process changes, and the process changes the mind, and the mind doesn't come back. In 2025, 55,000 layoffs cited AI as a direct cause. It's not that the beast was already doing those sailors' jobs — it's that the officers needed their wages to keep feeding it. Junior tech job postings dropped 67% between 2023 and 2024: people who were never given the chance to board the ship. And then there are those who stayed below, talking to the beast all day — people who sit down at ten at night with a mediocre idea and by three in the morning have something deployed and running. The backlog of years empties out. The dopamine is real. And they don't stop. They're the ones counting coins without knowing why, the ones copying maps of coasts that don't exist yet. Klarna announced that AI was doing the work of 700 support agents. A year later, its CEO admitted: "we went too far." They're rehiring humans. The beast spat out what it couldn't digest. For every function it absorbs, there's another it returns chewed up and useless.

But the uses each person finds are as different as the sailors' descriptions — a crab, a squid, a turtle. A diagnosis that catches a tumor the radiologist would have missed. Legal assistance accessible for the first time. Tools that save real hours for real people. Each one goes down to the hold and sees something different. Not pure gold — something in between. Enough that you can't ignore it. Enough that you can't trust it. If it produced nothing, it would be easy to stop feeding it. If it produced pure gold, there would be no dilemma. What it produces is exactly what makes this interesting rather than simple.

There is no neutral position 1. The developer who doesn't use these tools loses ground to the one who does. The company that doesn't adopt watches its competitors do so. The artist discovers that the market won't wait while they decide whether participating feels ethical. This article was written with the help of a model. There's no clean way to say that, but it seems worse not to. Is it a bubble? Could be — threads/the opinion bubble exists and is real, and the technology is too, and not being able to hold both ideas at once is perhaps the real bubble. Bubbles deflate. Structural transformations don't. And those who ask to pause until it's safe are asking for something that in practice means ceding direction to whoever doesn't pause. The problem with collective prudence is that it only works if it's truly collective, and that requires a coordination we've never achieved — not for climate, not for nuclear weapons, not for anything that actually matters. Harari said it with the clarity that things without solutions have: "we know this is very, very dangerous, but we can't trust the other humans."

1Upton Sinclair, 1935: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it." The mechanism hasn't changed.

Data centers consume 415 TWh per year, 1.5% of all electricity on the planet. Microsoft is buying nuclear reactors. Amazon is acquiring power plants. The biggest companies on earth are becoming energy utilities — to train language models, to predict the next word. Brands like Polaroid launch "made by humans" campaigns that generate 25% more engagement, and it's hard to decide whether that's resistance or whether protest, when it becomes profitable, stops being protest and becomes just another deck on the ship.

There is no moral. I said it at the beginning and I stand by it. It's not good or bad. It's an unknown — the biggest one, probably, of our time. Bigger than climate change, because with climate change we at least know what it is and what causes it. Not this. This is something whose nature is unknown, whose capabilities grow faster than our understanding, and whose ending isn't written. The ones counting coins are preparing for something they can't name. The ones playing cards are pretending nothing has changed. The ones who stayed in the hold talking to the beast can no longer tell whether they're using it or adapting to it. And the ones shouting from the stern — Hinton, Yudkowsky, the thirty thousand signatories — are right about what they describe and have no power over what they predict.

The story is being written. With every model trained, with every job that disappears and every diagnosis that saves a life, with every person who goes down to the hold and comes back changed. No one knows what the crab is. And that, for now, seems to me the only honest stance.