Hidden risks behind AI token supply management

Machine Learning


Businesses started treating tokens the same way they treat capital, tracking consumption, routing workloads to the cheapest model to do the work, and holding teams accountable for how much they spend. This discipline solves the cost problem. It doesn’t solve the entirely different problem hiding underneath. The same AI systems that run using compute tokens are also authenticated, connected, and orchestrated by an entirely different category of tokens: OAuth credentials, API keys, and machine IDs that allow models to communicate with data, tools, and each other. There is little point in managing token supply from a cost perspective if the tokens that hold the whole system together are left unmonitored, unrotated, and unaccounted for.

When trust becomes a target of attack

All AI deployments, regardless of their complexity, are actually a web of integration. The model calls the search system, the search system calls the vector database, and the agent calls the CRM. Each of these calls is authorized by a token that proves the request is legitimate. Palo Alto Networks Unit 42 security researchers describe these credentials as the invisible currency of trust in modern cloud environments, and that description applies just as much to AI pipelines as it does to traditional SaaS integrations.

The 2025 Salesloft Drift incident showed what happens when that trust is abused. The attacker, identified as UNC6395, stole a single OAuth token from a third-party integration and used it to completely bypass multi-factor authentication and gain legitimate and permanent access to hundreds of customer Salesforce instances. The attackers did not need to enter through the front door. The token was the front door, already unlocked, and the connected system had no reason to suspect the credentials, which seemed perfectly fine.

3 ways trust breaks down

Unit 42 research has identified a small number of recurring failure patterns behind nearly every major token-related breach, each of which corresponds directly to how AI systems are commonly built and connected.

The first is dormant integration. This is a connection that no one remembers exists, but remains fully permitted. In 2022, GitHub revealed that an attacker had compromised OAuth tokens issued for the Heroku and Travis CI integration, which many organizations no longer actively use. Authorization simply remained, and threat actors exploited that forgotten trust to gain access to private repositories and collect secrets. AI pipelines with dozens of connected tools, plugins, and data sources accumulate just these kinds of forgotten approvals as projects evolve and integrations are replaced without being formally revoked.

The second is insecure storage. CircleCI’s 2023 breach involved customer OAuth tokens, environment variables, and SSH keys stored unencrypted in the build environment, and once the attacker gained internal access, those credentials were simply stolen. CircleCI had to ask all customers to rotate all tokens and secrets they hold. This was a costly and destructive response to a problem that could have been completely prevented with better storage hygiene.

Third, there is no expiration date. The 2024 Internet Archive breach involved GitLab tokens that were valid for 22 months without rotation, giving attackers uninterrupted access for almost two years before exfiltrating 7 terabytes of data. A token without a lifecycle is a vulnerability with no expiration date.

Supply chain is deeper than credentials

Token compromise is just one layer of a larger problem. Workstream 1 of the Coalition for Secure AI published detailed guidance on how AI supply chains are fundamentally different from traditional software supply chains. The difference ultimately comes down to where the intelligence actually resides. In traditional software, sensitive logic resides in identifiable locations within the code. In an AI system, its intelligence is distributed across millions of weighted connections learned from training data. This means the attack surface extends to data that an organization has no direct control over.

This decentralized structure creates risk at every step. Training data can be contaminated by direct manipulation or by planting malicious content on public web pages that crawlers later ingest, and the resulting biases and backdoors can remain dormant until very specific inputs trigger them. Pre-trained models transferred between organizations carry the same risks in a different form, as the weights can be tampered with in a way that evades normal testing while in storage or in transit. Even the authentication systems that manage access to deployed models are themselves part of the supply chain. CoSAI’s framework specifically calls out API key management, token rotation, and role-based access control as core components of the model serving infrastructure. This means that the OAuth and credential risks mentioned above are not separate concerns from AI supply chain security. They are one of its central pillars.

Search extension generation systems pose further problems. Vector databases, embedded models, and the context assembly systems that feed the models each represent potential injection points, and a compromised vector database in a multi-tenant environment can leak information across customer boundaries in ways that are quite different from traditional data breaches.

Where does the government draw the line?

National cybersecurity agencies are beginning to treat this as a separate risk category rather than an extension of regular software security. The Australian Cyber ​​Security Center has published guidance specific to the AI ​​and machine learning supply chain, warning that the complex ecosystem of models, data, software libraries and cloud infrastructure behind modern AI systems can expose organizations to contaminated data, hidden backdoors and malicious code if left unmanaged. The agency’s guidance applies to any organization deploying or developing an AI system, whether that means building models in-house or simply leveraging a vendor’s pre-trained models or third-party datasets, and directs organizations to make AI-specific enhancements to their software bills of materials, also known as AI BOMs, as a way to make model components interrogable rather than treating them as opaque black boxes.

The guidance extends the supply chain discussion to hardware and recommends that organizations treat the underlying infrastructure of AI systems like any other integrated technology: enforcing signed drivers and firmware, enabling verified boot, and appropriately segmenting management networks from production systems. This is a reminder that AI supply chain security goes beyond models and data. It flows all the way to silicon.

Building a token governance program

Organizations that successfully manage this tend to focus on a similar set of practices, whether arrived at through security reviews or cost audits. The starting point is visibility. That is, a current inventory of all OAuth tokens, API keys, and service credentials in circulation, including which integrations they belong to and whether those integrations are still in active use. Dormant connections should be canceled on a regular schedule rather than left to accumulate indefinitely. Additionally, token lifetimes should be kept short by default, with rotation built into the system design, rather than being treated as an emergency response after a breach.

Storage discipline is just as important as lifecycle management. Tokens must be encrypted, not recorded in source code or logs, and must follow the same secret management standards that organizations apply to their most sensitive credentials. This is because a token stored in plaintext is only as secure as the weakest system that can read it. Runtime monitoring closes the loop as centralized logging and anomaly detection results in incident containment, rather than a breach where a compromised token goes undetected for months.

On the model and data side, the same discipline must be extended upstream. Documentation of provenance for training data and pre-trained model checkpoints, cryptographic signatures on model artifacts, and vendor assessments that require hosting providers to demonstrate their own security posture are no longer optional add-ons for organizations operating at scale. These are the differences between knowing what’s actually inside a model and simply trusting that the model will work as advertised.

outlook

Token-based competition in the cost sense rewards organizations that treat consumption as a resource to be measured and directed. Token security rewards organizations that treat trust in the same way, in the credential sense, as having an inventory, a lifecycle, and ownership. These two issues will continue to converge as AI systems connect to more tools, more data sources, and each other. And the companies that manage both sides of that ledger, spend and trust, will be the ones that can scale AI without quietly expanding their attack surface at the same time.

References

Batchelor, B., Rafian, E., and Quist, N. “Trusted Connections, Hidden Risks: Token Management in Third-Party Supply Chains.” Unit 42, Palo Alto Networks, September 12, 2025. https://unit42.paloaltonetworks.com/third-party-supply-chain-token-management/

“Establishing AI Supply Chain Risk and Controls, V 1.0” Coalition for Secure AI (CoSAI), Workstream 1: Software Supply Chain Security for AI Systems, June 12, 2025. https://github.com/cosai-oasis/ws1-supply-chain/blob/main/risks-and-controls-for-the-ai-supply-chain-v1.md

“Artificial Intelligence and Machine Learning: Supply Chain Risks and Mitigations” Australian Signals Directorate, Cyber.gov.au. https://www.cyber.gov.au/business-government/secure-design/artificial-intelligence/artificial-intelligence-and-machine-learning-supply-chain-risks-and-mitigations



Source link