Humanity has updated its Claude AI chatbot usage policy in response to growing concerns about safety. In addition to introducing stricter cybersecurity rules, humanity has designated some of the most dangerous weapons people should not develop using Claude.
Humanity does not highlight the tweaks made to weapons policies in a post summarizing the changes, but the comparison between the company's old and newer usage policies reveals a noticeable difference. Anthropology previously banned the use of Claude, but “producing, modifying, designing, selling or distributing weapons, explosives, dangerous materials, or other systems designed to cause the loss of human life” expands on this by specifically banning the development of high-yield explosives along with biological, nuclear and radiation (CBRN) weapons.
In May, humanity implemented AI Safety Level 3 protection with the launch of the new Claude Opus 4 model. Safeguards are designed to not only make jailbreaking models more difficult, but also to prevent them from supporting the development of CBRN weapons.
In that post, humanity acknowledges the risks posed by agent AI tools that allow Claude to control a user's computer, and the Claude code, a tool that embeds Claude directly into the developer's device. “These powerful abilities pose new risks, including scaled abuse, the creation of malware and the possibility of cyberattacks,” writes humanity.
AI startups are addressing these potential risks by folding the new “Do not compromise your computer or network system” section into their usage policy. This section contains rules for using Claude to discover or exploit vulnerabilities, creating or distributing malware, developing tools for denial of service attacks, and more.
Furthermore, humanity is loosening its policy on political content. Instead of banning the creation of any kind of content related to political campaigns and lobbying, humanity simply prohibits the use of Claude to “deceive or disruptive the democratic process or involves targeting voters and campaigns.” The company also revealed that the requirements for all “high risk” use cases work when people use Claude to make recommendations to individuals or customers, and apply only to consumer-oriented scenarios rather than business use.
