Artificial intelligence company Anthropic has warned that it may be approaching a major milestone in AI development: recursive self-improvement (RSI), a process in which AI systems help build and improve increasingly capable versions of themselves.
The warning appears in the company’s revised registration filing with the US Securities and Exchange Commission (SEC), submitted on Monday. While Anthropic says none of its current models has crossed its internal RSI threshold, it expects the milestone could be reached soon.
RSI refers to the possibility that AI systems could become capable of substantially accelerating their own development by helping researchers design, train and improve more advanced models.
The prospect has attracted both enthusiasm and concern. Supporters see the potential for major advances in productivity, scientific research and technological development, while critics warn that rapidly improving systems could outpace human oversight and existing safety measures.
Anthropic says AI-driven research could accelerate progress
In its filing, Anthropic acknowledged that its growing use of AI in research and development could cause model capabilities to advance faster than the company’s ability to develop and implement appropriate safety measures.
The company said it had previously failed to recognise significant improvements in its development pace until after they had occurred.
It also disclosed that internal estimates suggest it may soon meet the Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI”.
The filing states that the company’s internal assessments and expectations for future models, rather than its current systems, indicate that the threshold may be reached imminently.
However, Anthropic stressed that it has not designated any existing model as having crossed that threshold and is continuing to assess the available data.
What is recursive self-improvement?
Recursive self-improvement describes a potential cycle in which AI systems contribute to developing more capable AI systems, which can then help accelerate further improvements.
If this process becomes sufficiently effective, it could significantly increase the pace of AI development.
For AI enthusiasts, the prospect could unlock new capabilities in areas such as scientific discovery, engineering and complex problem-solving.
Safety researchers, however, have raised concerns that rapid advances could outpace human understanding and the development of safeguards. In particular, increasingly autonomous systems could become more difficult to monitor and control if their capabilities advance faster than oversight mechanisms.
Anthropic says humans will remain involved
Despite warning that the RSI threshold could be approaching, Anthropic said it does not expect to rely on Claude to independently design and build successive generations of frontier AI models without meaningful human supervision or understanding in the foreseeable future.
The company said it was considering changes to its Responsible Scaling Policy to ensure humans remain in control of frontier model development.
Its filing also cautioned that using AI to develop more capable AI systems could create considerable safety risks, particularly if it sharply accelerates technological progress.
The Responsible Scaling Policy is intended to guide how Anthropic monitors and addresses risks as its models become more capable.
Questions remain over how close AI is to self-improvement
The prospect of recursive self-improvement has become a prominent subject in debates about the future of artificial intelligence. However, defining and measuring the point at which AI can meaningfully improve its own development remains challenging.
Anthropic’s disclosure is based partly on internal estimates and expectations about future models, rather than confirmation that an existing system has achieved RSI.
The company’s SEC filing also serves as a formal disclosure to investors, making it important to distinguish its stated expectations from demonstrated technical capabilities.
Still, the warning highlights a growing concern within the AI industry: whether safety research, human oversight and governance can keep pace with increasingly capable systems.
