Nexigen, with support from UC, is working to ensure the security of AI systems through both input and output.
Governments, businesses, and other organizations want to protect personal data. Nexigen creates a secure repository outside of a large language model, an AI system that can process large amounts of data and generate content.
Through secure repositories, you can block sensitive data from entering large language models and being exposed outside your organization. UC's Cohen's expertise will help create more filters and increase accuracy and safety.
This partnership also helps ensure that the output from AI systems is relevant. This research aims to reduce “hallucinations” when AI systems produce non-existent or inaccurate outputs.
The purpose is also to prevent people from circumventing guardrails. For example, many AI systems prevent users from receiving instructions on how to do something illegal, such as building an atomic bomb. But users can sometimes get around these guardrails by requesting historical information, such as how the Manhattan Project built the first atomic bomb.
Currently, the biggest threat from AI is to the people who build and use the systems, but as programs become more sophisticated, they too could pose a threat, Salisbury said. Nexigen's partnership with UC positions us to combat these issues as well.
“We have a cybersecurity approach that is zero trust,” Salisbury said. “So we're looking for potential intent issues with the output on the LLM, so that if the LLM develops a sense of self-intelligence, we can actually detect it and kick it out of the network. I can.”
