Four years into the AI revolution, one thing has become abundantly clear. That said, while nearly all knowledge-based businesses use technology in some way, not all businesses know exactly how. This lack of oversight is not only a waste of resources and potential, but also contributes to an impending corporate security crisis.
Just as personal devices were compromised when smartphones became mainstream, “shadow AI” (employee use of AI within a company but beyond its reach) is flourishing. The difference this time is that employees are entrusting LLM with sensitive data. Sensitive data can easily be exposed directly or indirectly through cyber-attacks or AI tools retaining or exfiltrating information.
A recent Lenovo survey of 6,000 employees worldwide found that 70% use AI at least once a week, and companies were unaware of one-third of this activity. Meanwhile, an MIT-led research project says a “shadow AI economy” is already developing as staff lack trust in official AI initiatives and sanctioned, ring-fenced corporate tools, and are more comfortable relying on the LLMs they use in their personal lives to help create documents and organize workflows.
“Shadow AI is already widespread, largely because AI adoption is happening faster than most organizations’ ability to provide approved tools and clear guardrails,” said Dr Leanne Allen, UK head of AI at KPMG. She says those responsible are both novice users who prefer the convenience of mainstream LLMs for drafting and brainstorming, and experienced staff who pay for more sophisticated accounts that offer features beyond employer-driven tools.
“The typical shadow AI pattern is simple,” adds Allen. “Someone copies their work into a consumer-grade tool or uses ‘vibe coding’ to prompt an AI to quickly generate code, scripts, or prototypes outside of an approved development environment in order to work faster. This is often not malicious; it’s a routine optimization under time pressure.”
The use of shadow AI has already been blamed for various security breaches. For example, cloud provider Vercel said it may have suffered a loss of customer data because cybercriminals effectively took over an agent AI tool used by its employees, gained access to employees’ Google Workspace accounts through that tool, and gained access to employees’ Vercel accounts. Samsung has received three reports of sensitive information being accessed within a 20-day period, all of which were attributed to employees using ChatGPT..
“When employees use external tools, it becomes difficult to know what data has been shared, what output is trusted, and whether records exist for auditing or accountability purposes,” Allen says. Beyond data loss, she says, risks include leakage of intellectual property or copyrighted material, regulatory violations, and low-quality output that constitutes a work. Almost half of employees surveyed by KPMG in conjunction with the University of Melbourne said they knew that the use of AI in the workplace was against official policy.
According to the giants of the AI world, none of this should matter. Companies like OpenAI, the maker of ChatGPT, claim to remove sensitive information and put guardrails in place to prevent themselves or third parties from accessing anything their employees might upload. However, many AI experts are not convinced that LLM manufacturers fully understand the capabilities of their models.
An investigation by Imperial College London earlier this year found that it was technically possible for LLMs to stitch together parts of older versions of documents to create a new whole, despite a “de-duplication” process that was supposed to act as a safeguard. Igor Shilov, a Reich researcher who worked on the project, said tech companies are “concerned” about leaking personal information but clearly don’t understand the problem.
“Text is messy, and so is data. Sometimes the same thing is written over and over again, but worded slightly differently. There are also typos. Some documents have multiple changes. Over time, the chatbot may have seen dozens of versions of the same document with small changes,” says Shilov.
“We found that the model has what we call ‘mosaic memory,’ the ability to build memories by looking at overlapping parts of text and stitching them together. When you have multiple versions of the same document, they can differ by 10% here and there, but still contribute significantly to the memory of that document.”
This clearly represents a significant risk when employees use LLMs to iteratively create budgets, marketing plans, or business strategy documents, but it is not fully understood. IBM, which has been running a public relations campaign about the risks of shadow AI, along with IBM fellow and serial entrepreneur Jerry Cuomo, scathingly criticized LLM security measures in a recent study, saying, “Just because we’re not storing data intentionally doesn’t mean we’re not storing data unintentionally.”
This problem has been exacerbated by the shift from vibecoding to the use of tools such as OpenClaw. This will allow hobbyists to create AI agents that can perform increasingly complex tasks and exercise considerable initiative beyond the control of their creators. In an April report, Grant Thornton specifically warned of the new frontier of shadow AI that OpenClaw’s enthusiastic support among tech-savvy employees is creating, noting that “employees are turning to OpenClaw to manage communications, automate responses, and summarize emails. When using w, an AI agent may be configured to read incoming messages and take actions based on their content. This creates an opportunity for attackers to embed hidden or misleading instructions within legitimate-looking communications.
Allen adds that because agents are designed to be “like colleagues,” it’s natural for people to trust them implicitly and scrutinize their output less than in a typical LLM interface.
One response to the gradual penetration of shadow AI is to lean toward repression, prohibiting the use of unauthorized tools, and improving the way we centrally monitor AI use. But it risks stifling positive behavior and frustrating employees who want to take advantage of readily available technology to accelerate their work. Additionally, Shiroff adds, truly ring-fencing information is an expensive endeavor. “The type of data preprocessing that eliminates risk is more difficult and expensive than the commonly deployed deduplication approaches, so there is a trade-off.”
“Organizations are right to take shadow AI seriously as a risk, but they shouldn’t view it solely through that lens,” Allen says. “Shadow AI often indicates that employees are trying to use AI to work more effectively and are not finding what they need from the approved channels.
“This creates an opportunity. Instead of focusing solely on restrictions, companies need to think about how to make safe and effective AI truly accessible, with clear intended use cases and guardrails built into daily workflows. If organizations respond only with policies and prohibitions, they may miss the root reasons why the behavior is happening in the first place.”
That means gaining a deeper understanding of which tools are being used and how, and delivering a carefully curated set of officially sanctioned AI products that are secure, yet genuinely useful, and tailored to the needs of staff in different roles, she added. Prompts and warnings can be part of an information campaign to make employees aware of the risks of shadow AI, but must be accompanied by an acceptance that the use of AI will occur anyway. Responsible use should be “rewarded” in the same way that we currently recognize efficiency gains from technology.
However, that cultural shift can take time. There are concerns that AI could become a commodity that many companies do not own, as experts predict that cybercriminals themselves will be empowered and emboldened by access to AI.
Image credit: Worased Boontipchayakun and PixeloneStocker (Getty Images)
