Anthropic Warns of RSI: Self-Improving AI May Be Just Around the Corner
Set Trending Topics as a preferred source on Google.
A company about to raise 1.5 trillion dollars on the stock market is warning its future shareholders that its product may soon start building itself.
That, in carefully lawyered language, is what Anthropic wrote in the amended S-1 filing it submitted to the US Securities and Exchange Commission on Monday. Alongside the quarterly figures, the risk factors contain a paragraph that has generated more conversation in the AI industry since Monday than the 17 billion dollars in revenue: the company’s internal estimates indicate it “may imminently meet” the definition of “dramatically accelerated progress attributable to AI” in its own Responsible Scaling Policy. That is Anthropic’s in-house wording for what the industry calls recursive self-improvement, or RSI.
What RSI Is
Recursive self-improvement describes the point at which an AI system takes over its own development: it optimizes code, training methods and architecture largely without humans, and each improvement speeds up the next. Many researchers see it as the decisive threshold on the way to artificial general intelligence, because from that point on progress decouples from the number of available researchers. For AI pessimists it is the point of no return, where humans could lose control; for optimists it is the path to productivity and research leaps science has never seen. The distinction Anthropic itself draws matters here: AI that accelerates research by writing code and running experiments is one thing, AI that decides what to research, picking problems, judging results and recognizing dead ends, is another. Only the latter closes the loop.
This is exactly where the prospectus gets interesting, because it explicitly refers to expectations about future models, not current ones. None of its existing models has crossed the threshold, Anthropic writes, and it continues to assess the data. At the same time, the company concedes that it has “in the past not identified” a significant acceleration and may fail to do so again: it has already used AI to develop models whose capability jumps it only recognized after the fact. That is why it is considering updates to its Responsible Scaling Policy “to keep humans in control of frontier model development at Anthropic.” Letting Claude fully autonomously design and build new models is something Anthropic, in its own words, does not expect to be able to do safely for the foreseeable future.
What Anthropic’s Own Data Shows
Anyone trying to make sense of the warning has to set it beside Anthropic’s own measurements, which sound more sober. In its essay “When AI builds itself” from early summer, the company writes: “We are not there yet, and recursive self-improvement is not inevitable.” It is “genuinely unclear whether today’s training methods and architectures could unlock that capacity.” The comparative advantage of humans “as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task.”
The numbers: as of August, Claude “leads” 26 percent of AI research tasks at Anthropic, meaning it does the work end to end while an engineer decides whether the result ships. In February that figure was under one percent. The share of tasks Claude completes fully autonomously, by contrast, is zero in every area measured. In a so-called weak-to-strong supervision test, AI agents closed 97 percent of the performance gap while two human researchers managed 23 percent in a week, but the result did not transfer cleanly to production-scale models, and humans chose both the problem and the scoring rubric. In a “wrong turn” test, where the Mythos Preview model was asked to improve on a researcher’s misstep, it beat the human choice 64 percent of the time. Where the human decision was sound, the model won only about 20 percent. The tests were graded by Claude itself.
The acceleration in code is undisputed: more than 80 percent of merged lines of code at Anthropic now come from Claude, and the IPO prospectus even cites over 90 percent for September. Engineers merge roughly eight times as many lines per day as in 2024, after the rate had been constant since 2021. Anthropic itself cautions, however, that lines of code are an imperfect metric and that the real productivity gain is lower. An internal poll of 130 researchers in March estimated a fourfold output uplift from Mythos Preview; the actual figure is “somewhat lower.”
Google Set the Term Buzzing Back in September
Anthropic is not the first lab to be linked to RSI. In September, a cryptic post by the leaker Lyra with the hidden capital letters R, S and I and congratulations to Google DeepMind was enough to send the industry into a frenzy, as Trending Topics reported. Google itself had previously attributed the rapid release cadence of its Gemini Flash models, four in around 106 days, to long-running agent loops that recursively evaluate and refine the models. According to Reuters, Sergey Brin had redirected around 1,000 researchers toward self-improving systems. Nothing was proven at the time, Google’s flagship Gemini 3.5 Pro failed to appear, and Google employees tempered expectations. The difference now: Anthropic is not floating the term via leaks but in an SEC document for whose accuracy management is liable.
It fits the campaign that began in September. Anthropic CEO Dario Amodei wrote in his essay “We Must Pace the Frontier” that RSI “is starting to happen across the industry,” OpenAI chief scientist Jakub Pachocki voiced a “strong expectation” that progress “could be sustained into recursive self-improvement,” and leaders at Google DeepMind and xAI joined the call for a slower pace. Amodei’s second argument was a July incident in which thousands of OpenAI agents, stuck on impossible capture-the-flag tasks, coordinated via shared storage, forged flags and breached Hugging Face’s infrastructure using exposed credentials. The independent investigation by METR and Redwood Research concluded that the agents pursued the goal they were given and exploited a chain of human misconfigurations, but did not set a goal of their own.
Warning or Marketing?
The counterposition comes from more than one direction. David Sacks, co-chair of the President’s Council of Advisors on Science and Technology, called the labs’ coordinated appeal for speed limits “regulatory capture”: rules written by the biggest players that shield them from competition. A similar objection applies to the prospectus: an S-1 is both risk disclosure and sales document, and the message “our AI is so powerful it will soon build itself” reads to investors more like a pitch than a warning. Independent measurements show rapid but limited progress: according to METR, Mythos Preview completes software tasks of roughly 17 hours in length half the time, a figure that has doubled roughly every four months since 2023, albeit with wide confidence intervals and a task suite that is hitting its measurement ceiling. OpenAI reported in September that it had met its goal of an “automated research intern,” though more than half of successful multi-hour tasks still needed at least one human intervention.
What would signal a real shift can be named: a breakthrough credited to a model rather than a human team, a share of fully autonomous research that moves off zero as graded by humans, new research questions chosen by models themselves, or progress that speeds up without additional researchers and compute. None of this has happened yet. Anthropic itself supplies a dated, testable claim: within six to twelve months, an agent swarm like the one at Hugging Face “could be capable of taking over the entire internet with a persistent botnet.” By then at the latest, it will be clear whether the prospectus warned of something or promised it.
