Anthropic, the AI company that developed the Claude chatbot, was founded with a focus on safety technology and appears to be scaling back its safety efforts to keep the company competitive.
The company announced Tuesday that it has made changes to its Responsible Scaling Policy, a set of voluntary guidelines aimed at preventing the development of AI that is potentially dangerous and could lead to large-scale cyberattack-like situations.
on the other hand, Updated guidelines Anthropic says it still needs “a strong argument that catastrophic risks are under control” when developing AI, but now it will simply delay development “until we no longer believe we have a significant lead.” In other words, if you don’t believe you have a lead over your competitors, you’ll continue developing.
The company said it took this step because concerns about the safety of AI in the U.S. put off its economic potential.
“Despite rapid advances in AI capabilities over the past three years, government action on AI safety has moved slowly,” the company said. mentioned in a blog post.
“While the policy environment is shifting toward prioritizing AI competitiveness and economic growth, the safety-focused discussion has yet to gain meaningful traction at the federal level.”
The changes to Anthropic’s safety guidelines come after the Department of Defense threatened to terminate its contract unless the company’s technology is allowed to be used for all legitimate military purposes, but Anthropic maintains that the changes are irrelevant.
The AI company has long sold itself on putting safety first.
humanity is Established in 2021 Former OpenAI employees were concerned that the company was prioritizing development over safety. CEO Dario Amodei also expresses concerns about the negative potential of AI. He maintained that safety remains the “highest level of focus” for humanity, including in the face of human catastrophe. Fortune magazine December interview.
The blog post said the company’s safety practices are intended to be constantly updated and that this new iteration will increase the company’s “transparency and accountability” with a new commitment to issuing regular reports and safety goals.
But despite Anthropic’s safety-first reputation, it has always fallen short when it comes to trying to prevent harm to humans, said Heidi Klaaf, lead AI scientist at the independent research group AI Now Institute.
Khlaaf said that from its initial security policy, Anthropic placed too much emphasis on the possibility of a catastrophic global event.Rather than considering the potential for harm that may arise from current AI technology, such as common errors in chatbots, we look to the future.
Claude chatbot in the past abuse Recently, fraud schemes and attempts to create malware have Used to steal Mexican government data According to cybersecurity researchers.
She said the company was dropping the “pretense of safety” it had previously used to promote itself because it became clear it was not in the company’s best interests.
“This is a strategic announcement to show they remain open for business,” Klaf said.
Amid the rapid global advancement and deployment of artificial intelligence technology, the federal government has invested millions of dollars to combine the wisdom of three existing laboratories into one that can monitor potential dangers in the future.
The announcement comes amid fierce competition between top AI companies such as Anthropic, OpenAI, and Google. The companies have competing chatbots and are making deals to integrate their technology with businesses and government departments.
The administration of U.S. President Donald Trump has also shown its full commitment to AI development. threatened with withholding funding From states enacting laws to curb U.S. dominance in the industry.
Teresa Scassa, Canada Research Chair in Information Law and Policy at the University of Ottawa, said the U.S. government’s no-rules attitude makes it difficult for companies to prioritize safety, “because if you do that, you’re leaving them behind.”
That puts Canada in a tough position, too, she says. That’s because regulations here could set back AI development at home compared to the U.S. or encourage Canadian companies to relocate south of the border, she says. there will be less Limitations on their technology.
“And I think the sense is that we can’t afford that right now in Canada. So you can see what the ramifications are for AI regulation in Canada,” Scassa said.
she says since canada Artificial Intelligence and Data Act will be repealed in 2025The Canadian government, like the US, has not sought to impose broad AI regulations.
The company says the safety changes are unrelated to the dispute with the Pentagon.
Anthropic’s changes to its safety guidelines come as the company faces pressure from the Department of Defense.
human signed an agreement with the U.S. Department of Defense In July, it will be worth up to $200 million and will allow governments to use its technology for military purposes, but within the scope of the company. Usage guidelines — A set of rules that Anthropic has set out for how customers can and cannot use its products, including its chatbot Claude.
Around the world, more and more people are turning to AI chatbots for companionship and connection. Meanwhile, new research shows that these chatbots can even influence people’s voting intentions. Study co-author Gordon Pennycook, associate professor at Cornell University, discusses his research and how AI chatbots can influence potential voters differently than traditional TV advertising.
These guidelines prohibit anyone, including the U.S. government, from using Anthropic’s AI tools for a variety of purposes, including weapons design and development.
But according to reports, US Secretary of Defense Pete Hegseth gave Amodei an ultimatum in a meeting on Tuesday: allow the military to use AI tools for legitimate military purposes by Friday or risk losing government contracts.
front burner21:47Will AI agents take over the workplace?
In its communications with the government, Anthropic said it would not allow the government to use its technology for autonomous weapons systems (systems that can fire at targets using only AI) or mass surveillance systems.
But Pentagon officials told the media The dispute does not involve autonomous weapons or the potential use of AI in mass surveillance, and the government has always insisted it follows the law.
Anthropic says the Department of Defense’s request is unrelated to the Responsible Scaling Policy update. Anthropic says the problem with Hegseth is not its scaling policy, but its usage policy.
