Two new studies released Thursday show that one of the world's leading AI companies has a “unacceptable” level of risk management and “a prominent lack of commitment to many safe sectors.”
Even today's AI, the risks of entry for many top companies themselves can include AI that helps bad actors carry out cyberattacks or create biological weapons. Future AI models can escape human control completely, as scientists are worried about.
This study was conducted by nonprofit organizations Saferai and The Future of Life Institute (FLI). Each is second of its kind and hopes it will become a running series that encourages the group to incentivize top AI companies to improve their practices.
Max Tegmark, president of FLI, said:
read more: Some top AI labs have “very weak” risk management, according to research
Saferai assessed the top AI companies' risk management protocols (also known as responsible scaling policies) to score each company on its approach to identifying and mitigating AI risks.
In Saferai's assessment of risk management maturity, no AI firm was better than “weak”. The highest scorers were humanity (35%), followed by Openai (33%), Meta (22%) and Google Deepmind (20%). Elon Musk's Xai won 18%.
Two companies, Humanity and Google Deepmind, scored lower than the research conducted in October 2024. As a result, Openai surpassed Google in second place in Saferai's rating.
Saferai founder Siméon Campos said that despite doing good safety research, Google scored relatively low scores. The company also released a frontier model of the Gemini 2.5 earlier this year without sharing safety information in what Campos called a “bad failure.” A Google Deepmind spokesman said on time: “We are committed to developing AI safely and securely to benefit society. AI safety measures include a wide range of potential mitigations. These recent reports do not take into account all of Google Deepmind's AI safety efforts.
Humanity's scores have also declined since Saferai's last survey in October. This was partly due to a change in the company that added its responsible scaling policy a few days before the release of the Claude 4 model. This removed the commitment to tackling insider threats by the time humanity released a model of that caliber. “It's a very bad process,” Campos says. Humanity did not immediately respond to requests for comment.
The study authors also said the methodology has been in greater detail since last October, explaining some of the differences in scoring.
The company that improved its score the most was Xai, earning 18% compared to 0% in October. Meta scored 22% compared to his previous score of 14%.
FLI research was broader and looked at not only risk management practices but also corporate approaches to current harm, existential safety, governance and information sharing. A panel of six independent experts scored each company based on reviews of additional private data that were given the opportunity to be provided to the company based on reviews of published material such as policies, research papers, and news reports. The highest grade was scored by Humanity (c Plus). Openai won a C, and Google won a C minus. (Xai and Meta both earned D.)
However, in FLI's scores for each company's approach to “existential safety,” all companies scored below D. “They're all saying, we want to create a super intelligent machine that can cover humans in every way. And yet, they don't have a plan for how to control things like this,” Tegmark says.
