Cybercriminals could use inaudible background sounds in audio and video files to hack smart speakers and AI assistants and access personal information, a new study warns.
Modern voice assistants utilize AI tools called large-scale language models that tightly integrate voice and text.
Research has shown that carefully crafted prompts, known as “jailbreaks,” can circumvent safety guidelines and ethical restrictions built into AI assistants.
Hackers have been known to use jailbreaks to force AI chatbots to perform requests they are programmed to deny, such as generating hate speech, assisting in cyberattacks, or revealing restricted information.
Although text prompts have been widely studied, the security risks and operational implications of voice jailbreaks on AI systems have not yet been fully investigated, according to a team of cybersecurity researchers from China and Singapore.
These “hostile voices” that are undetectable to the human ear can trick AI models into doing tasks they aren’t supposed to do. “This study uncovers a previously overlooked threat: auditory prompt injection,” the researchers wrote in a non-peer-reviewed study posted on arXiv.
Hackers using audio jailbreaks can hijack the behavior of AI models by secretly providing limited input.
Although this type of attack is more restrictive than a text jailbreak, it can be “potentially more harmful,” the researchers said.
Researchers have developed a way to hijack voice-based AI models like smart speakers using imperceptible audio.
They tested this method. audio hijack, We tested 13 state-of-the-art audio-based AI models and found that the majority could be hijacked covertly, no matter what the user’s prompt was.
“This attack induces a variety of fraud behaviors, ranging from simple instant denials to complex tool misuse, with an average success rate of 79-90%,” the researchers said in their study.
“Hostile voices” can manipulate AI agents into performing unauthorized actions, such as downloading malicious files or leaking user information via email.
As on-device integration of AI becomes commonplace and widely deployed in electronic devices such as mobile phones and smart speakers, researchers warn that “there is no dedicated defense against this emerging threat.”
The researchers say the discovery reveals a fundamental vulnerability in AI models’ ability to integrate speech and text.
“In these settings, the insertion of auditory prompts can interact with system components and third-party apps, potentially enabling broader compromise,” they said.
“Future work should extend the assessment to system-level applications and real devices to better assess the actual risks.”
