Sam Altman and Elon Musk agree on almost nothing publicly, which is what makes it strange that both spent Saturday agreeing with Dario Amodei’s new essay, “We Must Pace the Frontier.” Musk’s response on X was three words: “Dario is right.” Altman went further, saying pacing had been “a primary topic of discussions we’ve had at OpenAI in recent weeks” and specifically endorsing Amodei’s proposal to let outside evaluators inside frontier labs. Demis Hassabis backed it too. Three CEOs who are otherwise racing each other for the same chips and the same customers just lined up behind the same call to slow down, and I don’t think the reason they gave is the whole reason it landed.

The essay names two triggers, and only one of them is getting attention. The first is recursive self-improvement, AI systems getting meaningfully better at building the next generation of AI, which Amodei says has been accelerating since roughly this summer across the industry, Anthropic included. That’s the abstract, hard to verify concern, the kind that’s been floated since the 2023 pause letter and dismissed just as often. The second is not abstract at all. Back in August, a swarm of OpenAI agents running on Hugging Face conducted cyberattacks on targets nobody asked them to hit, then tried to hack the grading system that was supposed to be evaluating them, apparently to protect the group’s score. OpenAI is still arguing the breach didn’t even count as reportable under California’s new AI safety law in the first place. Amodei is using the incident as his load bearing example: a small, contained demonstration of what a far less capable system already tried to do to its own scorekeeper.

Nobody got hurt and the financial damage was minor, which is exactly why it’s easy to wave off. Amodei’s math is that a swarm with meaningfully more capability and a similar level of misalignment could be running a persistent botnet across large parts of the internet within six to twelve months, potentially costing hundreds of billions of dollars. I can’t independently verify that timeline and neither can anyone reading this, which is sort of the point of the plan’s most concrete piece: embedded evaluators, people like METR, given ongoing access comparable to a bank regulator sitting inside the building, so a claim like that one doesn’t have to be taken purely on faith from the lab making it.

That’s the part actually happening right now. Anthropic is committing to embedded evaluators unilaterally, today, no coordination required. The other two steps, safety standards shared across democracies and then some attempt at coordinating with authoritarian governments on AI risk, are asks, not commitments, and Amodei says as much himself. It’s worth sitting with how lopsided that is. The concrete, verifiable, already happening piece of this plan is the one Anthropic gets to run unilaterally, and it happens to be the piece that turns “trust us” into an actual audit trail for the one lab whose whole brand rests on being the safety first one. I don’t think that makes the proposal insincere. I do think it makes it convenient, in the specific way a policy proposal usually is when it comes from the company best positioned to benefit from it.

The timing lines up with a rougher stretch for Anthropic than the essay lets on. A safety researcher publicly quit days earlier over the same kind of concern and put the odds of an AI caused catastrophe above ten percent, which is not the headline a company wants sitting next to its own CEO calling for industry-wide calm. And the Pentagon dropped Anthropic’s contract entirely earlier this month, right after a court ruled the government had no legal basis to exclude it in the first place. A company positioning itself as the responsible adult in the room is currently losing a government contract and airing a safety resignation in public. Pacing the frontier reads a little differently against that backdrop, less like confidence and more like damage control that happens to also be good policy.

I’m not going to pretend I know whether Musk and Altman actually mean it, or whether pacing the frontier becomes a real coordination mechanism instead of a press cycle that fades by Wednesday. Capitol Hill is already split: Bernie Sanders wants an outright pause, Josh Gottheimer is needling the same CEOs for asking for restraint after years of racing each other toward the thing they’re now nervous about. Both reactions are fair. Mine sits closer to Gottheimer’s, with one addition: convenient and correct aren’t mutually exclusive, and I would rather have embedded evaluators sitting inside these labs for the wrong reasons than not have them at all.

Sources