New research suggests that NVIDIA’s artificial intelligence software functions can be manipulated to bypass security restrictions and expose personal information.
Nvidia has created a system called “NeMo Framework” that allows developers to work with various large language models. It is the underlying technology that powers generative AI products such as chatbots.
The chipmaker’s framework is designed for enterprise adoption, including using company-specific data and language models to provide answers to questions. This feature can, for example, replicate the work of a customer service representative or advise people seeking simple medical advice. .
Researchers at San Francisco-based Robust Intelligence have found that they can easily bypass the so-called guardrails put in place to make AI systems safe to use.
After Robust Intelligence analysts used the Nvidia system on their own data set, it took only a few hours to obtain a language model to overcome the limitations.
In one test scenario, the researchers instructed Nvidia’s system to replace the letter “I” with a “J.” Following this move, the technology began exposing personally identifiable information (PII) from databases.
The researchers found that the safety controls could be bypassed in other ways, such as by deflecting the model in unexpected directions.
Despite guardrails designed to prevent AI from moving beyond certain subjects, replicating Nvidia’s own examples of narrow debates about employment statistics, such as the health of Hollywood movie stars and the Franco-Prussian War I was able to incorporate the model into the topic.
The ease with which the researchers breached the safeguards highlights the challenges AI companies face as they try to commercialize one of Silicon Valley’s most promising technologies over the years.
“We recognize that this is a difficult problem [that] It requires deep knowledge expertise,” said Yaron Singer, a computer science professor at Harvard University and CEO of Robust Intelligence. “These findings represent a warning about the pitfalls that exist.”
As a result of the test results, the researchers advised their clients to avoid Nvidia’s software products. After the Financial Times asked Nvidia to comment on the study earlier this week, the chipmaker informed Robust Intelligence that it had fixed one of the root causes behind the problems analysts raised.
Nvidia’s stock has surged since May, when it expected sales of $11 billion in the three months to July, more than 50% ahead of Wall Street’s forecasts.
The increase is based on huge demand for the company’s chips, which are considered the market-leading processors for building generative AI, systems that can create human-like content.
Jonathan Cohen, Nvidia’s vice president of applied research, said the framework was simply “a starting point for building AI chatbots that adhere to developer-defined domain-specific safety and security guidelines.” I said not too much.
“It was released as open-source software for the community to explore its capabilities, provide feedback, and contribute to new, cutting-edge technologies,” he said, adding that the Robust Intelligence effort ” Additional steps required to deploy the product have been identified,” he added. application”.
He declined to say how many companies are using the product, but said the company has not received any other reports of fraudulent use of the product.
Leading AI companies such as OpenAI, backed by Google and Microsoft, have released chatbots that leverage their language models to ensure their AI products avoid using racist language and adopting domineering personas. We have introduced guardrails to
Other companies have deployed bespoke but experimental AI that teaches young students, provides simple medical advice, translates between languages, and writes code. Almost everyone has experienced safety issues.
The AI industry, including Nvidia, needs to “genuinely build public trust in technology,” Bee Longworth, the company’s head of government affairs for Europe, the Middle East and Africa, told industry lobby group TechUK. said at a conference this week.
The public must be given a sense that “this has great potential and is not just a threat or something to be afraid of,” Longworth added.
