The largest and most influential artificial intelligence (AI) companies are joining forces to plan a security-first approach to the development and use of generative AI.
The Coalition for Secure AI (CoSAI) aims to provide tools to mitigate the risks associated with AI. Our goal is to create standardized guardrails, security technologies, and tools for the safe development of models.
“Our initial work will include security of the AI and software supply chain, and preparing defenders for the evolving cyber environment,” CoSAI said in a statement.
of Early efforts According to Google, one of the founding members of the coalition, its goals include building a safe bubble and a system of checks and balances on access and use of AI, and creating a framework to protect AI models from cyberattacks. Google, OpenAI, and Anthropic own the most widely used large language models (LLMs). Other members include infrastructure providers Microsoft, IBM, Intel, Nvidia, and PayPal.
“AI developers need, and end users deserve, an AI security framework that is relevant and responsibly seizes the opportunities before them. CoSAI is the next step on that journey, and you can expect more updates in the coming months,” wrote Heather Adkins, vice president of security engineering at Google, and Phil Venables, chief information security officer at Google Cloud.
AI safety comes first
Since the launch of ChatGPT in 2022, AI safety has raised many cybersecurity concerns, including the use of social engineering to infiltrate systems and the creation of deepfake videos to spread disinformation. At the same time, security companies such as Trend Micro and CrowdStrike have also begun to turn to AI to help enterprises eradicate threats.
Gartner analyst Aviva Litan said making AI safe, trustworthy and transparent is important because its results could lead organizations to misguided, sometimes harmful, actions and decisions.
“AI cannot operate in isolation without guardrails to restrain it — errors and exceptions need to be flagged and investigated,” Litan said.
AI security issues are likely to increase with technologies such as AI agents, which are add-ons that generate more accurate answers from custom data.
“You need to have the right tools in place to automatically remediate all but the most opaque exceptions,” Litan said.
US President Joe Biden has called on the private sector to prioritize AI safety and ethics amid concerns that AI could widen inequality and jeopardize national security.
In July 2023, President Biden issued an executive order requiring major companies currently members of CoSAI to develop safety standards, share safety testing results, and work to prevent the misuse of AI with biological materials and to prevent fraud and deception.
CoSAI will collaborate with other organizations, including the Frontier Model Forum, Partnership on AI, OpenSSF, and MLCommons, to develop common standards and best practices.
MLCommons told Dark Reading this week that it plans to release a suite of AI safety benchmarks this fall that will evaluate LLMs on their response related to hate speech, exploitation, child abuse, and sexual assault.
CoSAI is managed by OASIS Open, which like the Linux Foundation manages open source development projects. OASIS is best known for its work on XML standards and the ODF file format, an alternative to Microsoft Word's .doc file format.
