Organizations that can take full advantage of agent AI are those that clearly understand their threat models and design to counter them.
rob smith
Today’s AI assistants find answers. AI agent takes action. They can already read the codebase, write and edit files, run tests, and fix bugs autonomously from a single prompt. And over time, you’ll be able to use your credentials to handle everything from booking business trips to processing procurement requests.
It’s powerful. It is also a great responsibility. The AI Standards Innovation Center, a division of the National Institute of Standards and Technology (NIST), has become sufficiently concerned about the risks of agent AI to begin collecting information on how to track the development and deployment of these tools.
“AI agent systems can perform autonomous actions that impact real-world systems and environments and may be susceptible to hijacking, backdoor attacks, and other exploits,” NIST notes in the paper.
IT leaders need to understand not only what agents can do, but what that functionality means for their security posture. Agentic AI reshapes and expands the attack surface, greatly increasing the likelihood that vulnerabilities will emerge, such as interactions between agents, that traditional security models were not built to capture.
New features, new challenges
The nature of large-scale language models, especially agent AI, poses a variety of security challenges. Some are completely new, while others are twists on age-old issues.
One of the biggest risks is the core strength of AI bots, the ability to process natural language, combined with the agent AI’s ability to follow instructions. This combination enables “prompt injection” attacks, where the agent’s behavior can be manipulated by malicious instructions embedded in legitimate content or requests. The probabilistic nature of LLM further complicates this issue. The same prompt injection attack may succeed or fail in different attempts, making it difficult to comprehensively verify defenses.
The combination of functionality within a single agent also poses particular challenges. AI agents integrate language model inference and tool access, including the ability to read files, query databases, call application programming interfaces (APIs), execute code, and interact with external services. Risks do not arise from single capabilities, but from their combination and the ability of agents to perform these actions autonomously.
Agents with access to personal data, exposure to untrusted content, and the ability to communicate with the outside world present a significantly different risk profile than agents lacking any of these three elements. Some observers refer to this combination as “
Other risks include:
- unintended behaviorwhen an agent performs an action beyond its intended scope due to manipulation of misinterpreted instructions or prompts.
- privilege escalation, Here, an agent operating with broad privileges may perform sensitive operations beyond the intentions of the initiating user.
- cascading failuresCompromise of one agent in a multi-agent system can corrupt other downstream agents.
Dealing with risks
IT leaders and other federal employees can take concrete steps to mitigate these risks. Effective security requires layered controls at three levels.
- Model level: Use separate messaging roles and randomized delimiters to clearly separate system instructions from untrusted content. Secondary classifiers provide an additional layer, scanning inputs and outputs to detect injection patterns and unusual formats. These are risk mitigation measures rather than complete solutions, which is exactly why the layers underneath are important.
- System level: Enforce least privilege globally. Agents should access only the tools they need for their tasks using credentials that are narrow in scope and set to expire quickly. Examine the injection patterns of content entering the system and screen for content that leaves behind sensitive information such as credentials and personally identifying information. Enforce default deny network controls and restrict external communications to explicitly approved endpoints. Then, design a workflow that overcomes the “Fatal Three Elements.” Separating read-only and writable agents ensures that no single agent can access sensitive data, process untrusted content, or communicate with the outside world at the same time.
- Human supervision level: Require explicit approval for critical operations, and allow low-risk actions to proceed with notifications. Layering your approach prevents approval fatigue that can lead to oversight avoidance. Users should be able to stop execution at any time and, if possible, roll back partially completed work. When an agent acts on a user’s behalf, it records both identities and evaluates permissions at their intersection. Records all agent actions, timestamps, identifiers, tools launched, resources accessed, and results with enough detail to reconstruct events after the fact.
Opportunity outweighs risk
The risks are real, but so are the opportunities, and it would be a mistake to let one obscure the other.
Consider what it would be like when an agent works for you instead of against you. The right combination of data access, content processing, and external communications, when properly managed, makes agents truly powerful tools. AI agents can monitor systems, apply consistent security rules without fatigue, and respond to threats at a speed and scale unmatched by manual processes. They are power multipliers.
Human security teams are always needed. However, teams that deploy agents as a defensive tool have significant advantages over those that don’t: faster detection, faster remediation, and the reduction in human error that attackers expect.
Organizations that can take full advantage of agent AI are those that clearly understand their threat models and design to counter them.
Rob Smith is GitLab’s Vice President of Public Sector.
Copyright © 2026 Federal News Network. Unauthorized reproduction is prohibited. This website is not directed to users within the European Economic Area.
