If Microsoft, Google, and OpenAI refuse to share the parameters of their generative artificial intelligence platform, can we trust them?
Given the increasing acceptance of generative AI in society and the potential for this technology to fall prey to the bad guys, the culture of “big AI” default and secrecy sets a dangerous precedent. This is the argument developed by Baldur Bjarnason, who recently blogged that if AI is a black box, then corporate use of technology is vulnerable to new forms of “black hat his keyword manipulation.” I warned you.
“We’ve known for a long time that AI models can be ‘poisoned’,” Bjarnason wrote in a promotional essay for his forthcoming book. Even for large models, when this much data is required, attackers can influence the results produced by the entire system. “
He argues that it’s entirely possible that someone malicious has already poisoned ChatGPT, and that users have no way of knowing.
Bjarnason said he doesn’t really know, because OpenAI doesn’t talk about the language, the diffusion model, how the prompts used for training are validated, how the training dataset is scrutinized, or how the generated responses are fine-tuned. I was.
“[OpenAI’s] Confidentiality means we don’t know if ChatGPT is kept safe,” writes Bjarnason, author of The Intelligence Illusion.
Bjarnason said big AI companies, including OpenAI, the developer of ChatGPT, have refused to give impartial researchers access to their models and training data needed to reliably reproduce studies and studies. I am particularly concerned that He’s also concerned about the level of hype and false claims surrounding AI, much of which was seen in the industry at the RSA conference in San Francisco in late April.
“Assuming that a propensity toward pseudoscience and poor-quality research is not inherent in the AI research culture, we should take it for granted that an explosion of enlightened self-awareness will naturally turn the entire industry away from love. Even if you think, there are ridiculous ideas and exaggerated claims, but secrecy should still haunt us,” Bjarnason wrote. “This level of secrecy, or information asymmetry, is a fatal blow to the free market.”
Public interest security blogger Bruce Schneier posted one of Bjarnason’s recent essays on the Schneier on Security blog. This essay has aroused interest and comments.
Chatting about ChatGPT
One commenter, identified as IsmarGPT, argued that as a tech elite, we risk being adversely affected and that the public, fascinated by AI’s blind use, may learn a harsh lesson. . “AI is [the] It can cause more damage than good, but unfortunately it seems that way, mainly because of the short-term interest of the existing tech elite and the fascinated public, we find this difficult I’m going to find a way “
Another user, identified as Peter, said that despite ChatGPT’s buzz, skeptics see it as just a probabilistic tool that lacks real intelligence. “It’s only a matter of time before someone destroys his ChatGPT. The architect chose the probabilities for each successive word very wisely, but the probabilities remain random. No intelligence, no intention.”
Given the popularity and growth of generative AI, it’s understandable that there are strong opinions. The market for artificial intelligence hardware and services is expected to grow from $36 billion in 2020 to $90 billion by 2025, according to market analysis from IDC and Bloomberg.
“Businesses are keen to integrate conservative AI into their existing ecosystems. The share is expected to rise to 20% ($18 billion to $20 billion) by 2025.”
All Talk, Attack Vector
AI data poisoning, a type of adversarial attack, involves attackers manipulating training datasets by injecting poisoned or tainted data, controlling the behavior of trained machine learning models, and causing erroneous Occurs when providing results.
In a recent essay, Bjarnason details 12 cases of compromised AI models, with attacks on almost every kind of AI model.
“It doesn’t seem to require any special knowledge of the internals of the system,” Bjarnsason writes. “Black box attacks have been proven time and time again to work, which means OpenAI’s secrecy is useless.”
Krishna Vishnubhotla, Zimperium’s vice president of product strategy, said that while attackers can’t beat ChatGPT overnight, they can slowly attack over time. Vishnubotra said the change to the “broken” state is so subtle and happens over time that it’s impossible to know exactly when it happened.
Do you bark louder than you chew?
“OpenAI should consider making its models accessible to the research community,” Vishnubotra said. “The ability to collaborate and share knowledge will facilitate progress in different areas of society. , can mitigate the spread of misinformation. Such powerful and essential technology should serve the greater good of humanity rather than be monetized solely by the private sector.”
Netenrich’s chief threat hunter, John Bambenek, said that ChatGPT is mostly harmless in its current state. As with any tool, risk becomes significant when people attempt to use it to perform some work or task.
“Right now we are in the novelty stage,” Bambeneck said. “For example, when Facebook introduced facial recognition, it was also a novelty. AI/ML systems will always need a human on board, and if you take the human out, you’re going to break it down for the worse, because it applies to more important problems than high school students trying to cheat their papers. ”
Bambeneck added that no one really knows how to regulate or ensure the security of AI/ML systems at this point. One idea is to treat it like cryptography with full transparency and critical review. That’s a big draw, but he worries that the more open AI is, the easier it will be, not the harder it will be to attack.
To date, the European Union has passed draft regulations that severely ban AI systems that pose an unacceptable level of risk to people’s safety. This includes systems used to deploy subconscious or intentional manipulation techniques, exploit people’s vulnerabilities, or classify people based on attributes. Social behavior and socioeconomic status.
In the United States, efforts are underway to apply the principles of fairness, nondiscrimination, and safety to AI products for use by federal agencies, and President Joe Biden has publicly commented on the potential benefits and dangers of AI. I’m commenting, but nothing serious. National legislation on AI has emerged.
