NEW YORK – Artificial intelligence company Anthropic on June 4 proposed a global pause on building its most powerful AI systems as its latest models begin to show signs of escaping human control.
The company, which makes the Claude family of AI models, said in a report that a global slowdown in cutting-edge AI development is “probably a good thing”, but warned that if just one company stands still, rivals will only compete.
“We believe the world is better off with the option of slowing or temporarily halting frontier AI development so that social structure and coordination research can catch up with technological advances,” the group said.
According to Anthropik, actually ceasing operations would mean that several large AI companies in multiple countries (particularly the US and China) would agree to cease operations at the same time under rules that everyone can actually verify.
This idea may be somewhat unpopular with people like Elon Musk, as the long-awaited stock market debut of SpaceX, which owns his artificial intelligence venture xAI, is expected to make him the world’s first trillionaire.
“Without a global coordination mechanism, businesses and governments will have to make difficult safety decisions in the face of competitive and geopolitical pressures,” Antropik said.
The company has faced pushback from industry insiders and White House officials, who argue that focusing on worst-case scenarios exaggerates risks and amounts to a strategy to slow down competitors using safety concerns as cover.
The company has faced pushback from industry insiders and White House officials, who argue that focusing on worst-case scenarios exaggerates risks and amounts to a strategy to slow down competitors using safety concerns as cover.
Still, the White House recognizes the company’s power. myth model – Due to its cybersecurity features, it is not publicly available and is currently deployed only in a small number of vetted organizations.
The proposal will face a tough battle in Washington and Silicon Valley. U.S. officials and technology executives have repeatedly argued that slowing AI development risks giving China a decisive strategic advantage in what many see as the defining technology race of this century.
However, US President Donald Trump said during his recent visit to Beijing that he discussed the possibility of cooperating with China on AI safety issues.
President Trump also signed an executive order this week allowing the government to conduct a 30-day preliminary review of the most powerful American AI model before announcing it.
Antropic likened the issue to nuclear arms control treaties, but said AI training would be much easier to hide than missile silos and even harder to address because the temptation to keep quiet is so great.
“We need the option to take our foot off the gas and hit the brake,” Anthropic co-founder Jack Clark said on BBC Newsnight in the UK on June 4.
“Currently, the AI industry has a gas pedal but no brake pedal.”
The company said it plans to convene government officials, scientists, advocacy groups and competing AI companies in the coming months to figure out how such a system would work.
Anthropic said this call for adjustment is also accompanied by internal data showing that AI is already dramatically speeding up its own development.
This acceleration could create a feedback loop that could eventually lead to what researchers call “recursive self-improvement,” Antropic warns.
This is the idea of AI systems that can essentially learn on their own and become smarter, with little to no human help.
“We are not there yet, and reflexive self-improvement is not inevitable,” the report said, but added that it could get there sooner than most governments and institutions are ready.
“Evidence suggests that the role of humans at each stage of the AI development process is narrowing.” AFP
