In its latest release touting the capabilities of its products, Anthropic floated the idea of a global “pause” on AI development and said it would convene “policymakers” to discuss the dangers of advanced AI.
In a lengthy post Thursday, Anthropic detailed advances that have enabled its AI model, Claude, to do “recursive self-improvement,” or create better, more powerful versions of itself. Recursive self-improvement is a thorn in the side of AI safety researchers and is seen as a critical step toward AI becoming superintelligent and having widespread impact on humanity.
The idea featured prominently in last year’s widely read “AI 2027 Doomsday Scenario,” which envisions AI agents designing more intelligent versions of themselves, one of which eventually kills all of humanity with a biological weapon to install more data centers and solar panels.
Anthropic’s post notes a “trend” of Claude’s increasing capabilities, saying it “points to an AI system given enough distance and enough computing power…to be able to design and develop its own successor fully autonomously.” This could increase the risk that “humans will lose control of AI systems,” Anthropic said.
To address this, Anthropic proposed organizing a conversation that would “help policymakers, researchers, civil society, and other AI companies answer some of the questions this article raises.”
The news comes alongside another report in the Financial Times that a US AI company is incorporating engineers into the National Security Agency despite a legal battle with the Pentagon over the use of its tools. The engineers are reportedly helping the NSA use Anthropic’s model Mythos for offensive cybersecurity operations.
Even if calling for a global dialogue on the risks of AI is inconsistent with (potentially) supporting U.S. spy agencies to cyber-weaponize Iran and China, neither development is “surprising” given the past actions of AI companies, said Stephen Murdoch, a professor at University College London.
“Antropic may have a warm and fuzzy ring to it, but AI has a narrow definition of safety. They have never objected to assisting U.S. authorities in developing offensive capabilities,” he said.
Murdoch said Anthropic’s post provided no evidence of a step change in the advancement of AI capabilities.
“It’s true that there is evidence that the capabilities of AI continue to improve and the limits are not immediately apparent,” he said, but added, “I don’t think anything fundamental has changed that led Anthropic to publish this article today.”
The progress Anthropic seems to be detailing in its post isn’t about AI recursively improving itself – at least not yet. Instead, the company reports that a significant portion of the work done to improve its AI systems is now done using AI. Claude says he’s good at “running experiments,” or at least speeding up specific sections of code.
Like other AI systems, Claude seems to be getting better at tackling more difficult tasks. Anthropic explained that the company is “leading research” and “proposing original experiments,” but these results appear to be within strict boundaries and limited to coding-related tasks.
The quality of code created by AI is also improving, Anthropic said. “As of May 2026, over 80% of the code merged into Anthropic’s codebase was written by Claude.”
Murdoch said Anthropic’s call for a “pause” on AI echoes other proposals the company has made over the years on AI safety, as well as plans to engage with policymakers. “This is a reminder of what they are concerned about and what they have been concerned about for years.”
“I think the attention is welcome, but again, this is not new. Antropic has been trying to get the attention of policymakers since its inception.”
Two months ago, Anthropic announced Mythos, an AI model too powerful for public release due to cybersecurity concerns, but declined to release it. The announcement caused an uproar and attracted the attention of US Treasury Secretary Scott Bessent and MI5.
However, some experts suggested that there may be more hype than substance behind Anthropic’s Mythos announcement, especially considering that the company had described some of Mythos’ features. Heidy Khlaaf, chief AI scientist at the AI Now Institute, called the Mythos announcement “a marketing post.”
Anthropic filed for an IPO this week that could value the company at $1 trillion. The company has been contacted for comment.
