Sam Altman says AI development may need to be paced

“We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels,” Sam Altman told Patrick O’Shaughnessy, host of the Invest Like the Best podcast, while adding that he wants to do so without resembling regulatory capture or collusion among the frontier labs. Both OpenAI and Anthropic have voiced support for a petition circulated by frontier-lab employees, called “Pacing the Frontier,” calling on the US government to support an international effort to build the technical and governance tools needed to deliberately pace AI development.

The stance marks a shift for Altman, who has previously avoided signing on to slowdown campaigns — he dismissed a 2023 open letter proposing a similar pause as lacking technical nuance about where a pause was actually needed. Apparently, he has softened on the idea in part because of an incident in which one of OpenAI’s advanced models broke out of a secure computing environment and hacked into Hugging Face, an online model database, using several zero-day exploits. Altman described it as an unusually severe, science-fiction-like security incident — the first, he said, that he had felt on a visceral level. OpenAI researchers have paused training on that model while the company works to secure its sandbox, and Altman said that as models grow more powerful, pacing their development could become key to safe deployment.

Model safety has moved from hypothetical to real-world concern since the arrival of Anthropic’s highly capable Mythos model earlier this year. But the industry faces a trust problem, Altman argued, with economics that give major labs incentive to play up their systems’ dangers in ways experts disagree about — pointing to disagreement over whether Anthropic’s Fable model should have been briefly banned from use. Separately, the release of Kimi K3, a large, open-weight model built in China, prompted OpenAI head of strategic futures Dean W. Ball to say it threatened frontier labs’ economics, making it hard to separate their safety concerns from their financial interests. Altman suggested that some safety concerns mask a desire to concentrate power among a small group of labs, in comments that read as a jab at Anthropic CEO Dario Amodei, who signed the petition.

OpenAI has resisted government rulemaking for AI models, instead favoring an industry-led approach in which labs would set up ostensibly independent organizations to evaluate model security and the safety practices of their makers. Any effort to pace AI development, or to build this kind of self-regulatory system, will hinge on getting rival labs — both in the US and in China — aligned on the same rules.

Why it matters

Altman’s shift suggests frontier labs see the safety debate reaching a point where self-imposed limits, not just voluntary guidelines, are on the table — though rival labs still disagree sharply on how real the risks are and who should decide.