Everywhere, organizations and employees are rushing to use AI to increase productivity and tackle the work of ROTE, but new research shows that this could prove disastrous. Experts say malicious actors can use AI exploits to access sensitive data, especially if the target does not have the proper AI governance and security controls in place.
IBM's 2025 Data Break Report Costs found that 13% of organizations have experienced recent breaches, including AI models or applications. Half of these (over 60%) said the incidents led to broad data compromises, with one in three reporting operational disruptions. Despite AI security and governance lagging behind adoption rates, attackers concluded that AI is viewed as a high-value target. Meanwhile, one in six data breaches were linked to AI-based attacks.
This week's featured highlights the potential for AI exploits and the importance of taking steps to protect AI, such as creating AI security policies and implementing AI governance. Read more about IBM's research and learn how AI exploits can hurt your company.
“Prompt Man” Attack can target chatgpt and genai tools
Layerx researchers have demonstrated the possibility of using the “man in the prompt” attack. This Exploit can use the browser extension's ability to access the Document Object Model (DOM) to read or inject prompts from AI tools without any special permissions.
Attackers can deploy malicious extensions through a variety of traditional methods, such as social engineering and purchasing access to legitimate extensions, but they can steal sensitive data from both commercial and internal LLMs. In-house LLMs are particularly vulnerable due to their often own data and lack of security guardrails.
Layerx CEO and co-founder or Eshed called this attack vector “very low fruit” because traditional security tools often lack visibility into DOM-level interactions.
Read the complete story by Alexander Claffy about Dark Reading.
Shadow AI increases the cost of data breaches
IBM's annual data breach investigation suggested that unsupervised shadow AI could increase average $670,000 per violation. One in five organizations reported experiencing cyberattacks, at least in part, related to Shadow AI. 97% of AI-related breaches occurred due to lack of proper access control.
Supply chain intrusions through compromised apps, APIs, or plugins were the most common way to access Shadow AI tools.
Despite the increased risk of Shadow AI, 63% of breached companies lacked AI governance policies. Even those with policies often failed to implement approval processes or strong access controls, with only 34% of whom regularly checking for the use of unauthorized tools.
At the same time, hackers were increasingly using genai for phishing and deep falk spoofing attacks.
Read Eric Geller's complete story about cybersecurity diving.
LLM that can emulate sophisticated attacks
Researchers at Carnegie Mellon University, a partner with humanity, have demonstrated that LLM can autonomously carry out sophisticated cyberattacks without human intervention.
The researchers created an attack toolkit called Incalmo, which uses the same cyberattack strategy from the 2017 Equifax CyberTack. LLM provided high levels of strategic guidance, while LLM and non-LLM agents performed low-level tasks such as exploit deployment. In nine of the 10 tests across small enterprise environments, Incalmo managed to remove some sensitive data, said researcher Brian Singer.
Researchers explained that it is not clear how well Incalmo works on other networks, and how effective it is for modern security controls. Still, the singer expressed concern about the speed and low cost of such attacks, noting that human-run defenses may suffer from machine threats.
Read David Jones' complete story about cybersecurity diving.
Editor's Note: The editors used AI tools to help generate this news brief. Our expert editors should always review and edit content before publishing.
Kyle Johnson is the technology editor for Informa TechTarget's SearchSecurity site.
