Anthropic's CEO says policy can't keep pace with AI
Anthropic's CEO says AI is moving faster than government can keep up, and lays out what he wants done about it.
This is the head of Anthropic, the company whose models you use every day, laying out in his own words where he thinks this is headed and what could go wrong. Dario Amodei opens with an image from The Lord of the Rings: two hobbits try to convince Treebeard, a slow and ancient tree, to defend his forest from an army cutting it down. Amodei says AI and government now have the same mismatch: AI moves fast, policy moves at Treebeard's pace, and the gap between them is where the real danger sits.
He credits the pace of progress to what he calls scaling laws, the pattern where AI models get reliably smarter as you feed them more computing power, meaning the chips and time needed to run them, a pattern now backed by more than a decade of evidence. If that pattern holds for just one or two more years, Amodei says we get what he calls Powerful AI, which he describes as "a country of geniuses in a datacenter."
What pushed him to write this now, he says, is that the risk stopped being theoretical. He points to Claude Mythos Preview, an early version of Anthropic's most capable model, and the discovery that the most powerful AI models can pose real cybersecurity risks, with the potential to disrupt the financial system, critical infrastructure, and national security. He says Mythos Preview scrambled the global cybersecurity landscape, and that its bigger significance is proving AI models are now tools of real global and national consequence, not consumer novelties.
Alongside the essay, Anthropic is releasing two concrete proposals: draft legislation on testing the most powerful AI models before they are released to the public, and a policy framework for job displacement, which Amodei says Anthropic intends to back with substantial money of its own. He writes mainly about US policy, since Anthropic is an American company, but says most of it applies elsewhere too.
He is not arguing for regulation for its own sake. He says that is exactly why Anthropic has spent the last few years pushing lighter, safer bets: rules requiring companies to disclose what they are building, limits on chip exports, and better data on how AI is changing jobs. He now thinks that was not enough, and that policy is still struggling to keep pace with where the technology already is.
a country of geniuses in a datacentervia Dario Amodei →