There is no longer any need to dare to have a completely different view of humanity.
The AI startup founded by former OpenAI employees is focused on developing the technology properly, but its fundamental safety principles are being undermined.
In a statement on Tuesday, Anthropic said it would no longer honor its commitment to “pause the scaling up of new models or delay the rollout” when such advances could outweigh the company’s safety measures amid increased competition and a lack of government regulation.
The new policy means Anthropic is far less constrained by safety concerns at a time when its flagship chatbot, Claude, has disrupted financial markets and raised concerns about the software dying.
As part of the changes, Anthropic now has separate safety recommendations for itself and the AI industry as a whole called a “Responsible Scaling Policy.” This policy was loosely modeled after the U.S. government’s Biosafety Level (BSL) standards.
Jared Kaplan, Anthropic’s chief scientific officer, told Time magazine that responsible scaling policies don’t fit with the current state of the AI race.
“We felt that if we stopped training the AI model, it wouldn’t actually help anyone,” Kaplan told Time. “With rapid advances in AI, we didn’t really feel it made sense to make a unilateral commitment when our competitors were making strides.”
An Anthropic spokesperson said in a statement that the new policy, which requires public risk reporting every three to six months, is “the strongest level of public accountability and transparency yet.”
“From the beginning, we have said that given the speed of AI and the uncertainty in this area, we needed to iterate and improve our policies quickly,” a spokesperson said in a statement to Business Insider.
The new policy still includes a promise to delay the development or release of “highly capable” AI models only in more limited circumstances.
In a lengthy blog post, Anthropic cited an “anti-regulatory political climate” as part of the reason for its decision. The company and its CEO, Dario Amodei, have pushed for AI regulation with some success at the state level, but no major action has been taken at the federal level.
“We continue to believe that effective government engagement on AI safety is necessary and achievable, and aim to continue to advance the dialogue based on evidence, national security interests, economic competitiveness, and public trust,” the company wrote. “However, this is proving to be a long-term project and is not something that is happening organically as AI capabilities improve or pass a certain threshold.”
The company said its scaling policy was always intended to be a “living document” and was outlined in its first version in 2023. However, Amodei has previously said that the safety policy is aimed at mitigating the risks that AI may pose, even quoting Uncle Ben’s famous admonition to Peter Parker, aka Spider-Man.
“The power of the model and its ability to solve all these problems in biology, neuroscience, economic development, governance, and peace that are so much a part of the economy comes with risks, right?” Amodei told podcaster Lex Fridman in November 2024. “With great power comes great responsibility.”
Anthropic said another reason for changing the standard is that high-theoretical risks of ASL-4 and above cannot be contained by one company alone using the framework. (In the world of biosecurity, BSL-4 refers to the highest level of protection that a very small number of laboratories put in place to work with pathogens like the Ebola virus.)
Safety is at the core of Anthropic’s soul
Amodei has repeatedly said his company’s commitment to safety is reflected in one of its first major decisions: to postpone Claude’s release to the summer of 2022.
Reflecting on the move, Amodei said he was concerned that Anthropic would not be able to develop safeguards fast enough to release its breakthrough technology to the public. OpenAI released ChatGPT in November 2022, igniting the AI race. After several months, Anthropic finally released Claude.
“This was commercially very expensive,” Amodei said in a recent interview with billionaire investor Nikhil Kamath. “That’s probably why we established our lead in consumer AI.”
The policy change also comes as Anthropic faces pressure from the Department of Defense over red lines it has set regarding its use of AI models. Amodei met with Defense Secretary Pete Hegseth on Tuesday to discuss the issue. Anthropic faces a Friday deadline, or Hegseth could reportedly seek to use his powers to force the company to withdraw.
One of Claude’s previous training documents, known internally as a “soul document,” is an example of rhetoric that would be out of place at most other AI companies.
Mr. Kamas asked Mr. Amodei how he would respond to critics who say Anthropic is simply pushing for regulation to prevent the growth of future competitors. Amodei said the 2022 decision is an example of how the company supports the safety debate. He also pointed to the United States’ insistence on restricting exports of advanced chips to China, a position criticized by Nvidia CEO Jensen Huang.
“For anyone who thinks we’re the only ones who would benefit from doing that, it’s very hard to think of a situation where that would be the case,” Amodei said. “You look at any of these and think, ‘Okay, okay,’ but you put them together enough. I don’t know. Judge us by our actions.”
