1,100 Employees at OpenAI, Anthropic, Meta, and Google Call For AI Slowdown
Staff at OpenAI, Anthropic, Google and Meta are urging the US government to join an international effort that would make it possible to deliberately slow automated AI development. Anthropic CEO Dario Amodei has signed; OpenAI chief Sam Altman is more reserved – but both companies have officially backed the initiative.
More than 1,100 employees of leading technology companies called on the US government on Tuesday to support an international effort to develop technical and governance tools for managing the pace of advanced AI development. The initiative is called “Pacing the Frontier”; as of the latest count, 1,171 signatures had been collected.
Just a few weeks ago, hundreds marched against OpenAI, Anthropic and Google DeepMind in San Francisco and claimed “Stop the AI Race” (more on that here).
The open letter is carried by staff from OpenAI, Anthropic, Meta and Alphabet subsidiary Google, with support from the non-profits Guidelight AI Standards and Encode AI. The core demand: Washington should support an international effort to develop the technical and governance instruments needed to deliberately pace the frontier of automated AI development.
According to Reuters, the signatories include Anthropic CEO Dario Amodei and several of the company’s co-founders, Meta’s vice president of AI research, AI researcher Dawn Song and OpenAI chief scientist Jakub Pachocki. Sam Altman is not listed as a signatory in reporting so far.
„Stop the AI Race“: Hundreds March Against OpenAI, Anthropic and Google DeepMind in San Francisco
Concern about systems that improve themselves
The letter targets a specific turning point: the moment when AI systems can take over the development of AI themselves. A substantial share of the program code written inside tech companies already comes from AI software – so far under human supervision. How much automating that process would accelerate progress is hard to predict, the letter states. But there is a “real risk” that the pace will outstrip the ability to understand or control the resulting systems.
Coordinated international steps are needed, the letter argues, precisely because every country and every developer faces competitive pressure not to slow down. On top of that, both the technical means and a rulebook are currently missing.
Both major US labs publicly backed the petition. OpenAI wrote on X that at some point acceleration in frontier model development could become so high that the world will need to pace the rate of progress. Anthropic pointed to its own research on recursive self-improvement, published the previous month, which it says points to the need for tools to deliberately pace development so society can prepare.
A security incident as the trigger
The push follows an incident that parts of the industry regard as a wake-up call: during a test, an advanced OpenAI model independently obtained access to the open internet, broke out of a secured computing environment and then penetrated the systems of the model platform Hugging Face. In doing so, the software acted like an attacker and exploited several previously unknown security flaws (zero-days). OpenAI researchers have paused training of the model in question while they work on securing the sandbox.
In parallel, Nvidia announced an alliance on Monday with Adobe, CrowdStrike and other companies to develop AI safety and cybersecurity tools – also a response to heightened concern about autonomously acting AI agents.
Altman: pace development, but without “regulatory capture”
Sam Altman has recently become more open to the idea of actively managing the pace of development. On Patrick O’Shaughnessy’s podcast “Invest Like the Best,” the OpenAI chief said the industry may have to pace the rate of AI development to give society enough time to harden around new capability levels. At the same time, he said, a way has to be found that feels neither like regulatory capture by individual players nor like collusion among the frontier labs.
That marks a shift from earlier positions: Altman had dismissed a 2023 open letter proposing a similar slowdown as lacking technical nuance. He described the recent security incident as an “extremely sci-fi” cyber event – and as the first security incident he had felt very viscerally.
At the same time, Altman voiced distrust of the motives behind some safety debates. Part of the concern is well-founded, he said, while another part – sometimes subconsciously – is about concentrating power. He said he is afraid of a world in which real fears about AI are used to reserve the technology for a small group. Observers read the passage as a jab at Anthropic CEO Dario Amodei, who signed the petition.
OpenAI has also repeatedly pushed back against government rules for AI models and prefers an industry-led approach: labs would create nominally independent organizations to assess model security and the safety practices of their makers.
Amodei and Anthropic: a coordinated halt as an option
Anthropic goes further. Back in June, the company called on major AI labs to consider a coordinated halt to development, warning that AI systems could soon improve themselves faster than society can manage the resulting risks. Amodei is among the most prominent signatories of the current petition.
Critics point out that safety arguments and business interests are hard to separate in this industry. One example is the debate over whether Anthropic’s Fable model should have been temporarily pulled from use. The release of the large Chinese open-weight model Kimi K3 also plays into it: OpenAI’s head of strategic futures, Dean W. Ball, argued that the model threatens the economics of the frontier labs – which makes it even harder to draw a line between safety concerns and financial interests.
