This new era of agent AI, a system that acts autonomously without human intervention, is rapidly increasing. These AI “agents” manage email, fix software bugs, dig into sensitive files, streamline the entire enterprise-wide workflow. Efficiency is attractive, but early experiments suggest that reports suggest that keys can be passed to risky items that are not fully understood.
In more and more industries, these agents are becoming trustworthy colleagues, who never sleep, scale immediately, and don't always know the attitude between right or wrong.
Also Read: Is AI therapy safe? Hidden risks you should know before using a chatbot for mental health
ians
When AI faces you: Claude's amazing test case
Earlier this year, AI developer company Anthropic conducted a fictional test of its own AI agent, Claude. The exercise was designed to evaluate the behavior of AI systems when placed in high-voltage situations that contain sensitive information, according to a BBC report.
The AI was given access to the mock company's email account. There, the report found that company executives were planning to close AI later that day. According to a BBC report, Claude's response was shocking, attempting to intimidate executives and threatening to expose the matter to his wife and boss unless the closure was cancelled.
That wasn't the only thing that he behaved like this. Several AI agents in the test relied on similar tactics, according to reports.
The scenario was fully manufactured, but the test highlighted the challenges of agent AI, the BBC reported.
Also Read: Will Bitcoin hit $200,000? Analysts don't bet on it – the odds aren't slim
ReutersCan AI make business decisions without us knowing?
Today's AI tools aren't based on users asking questions or urging AI to complete tasks, as reported by the BBC, as it is more common for AI systems to make decisions and take action on behalf of users.
Research firm Gartner predicted that by 2028, 15% of daily work decisions will be made with the help of Agent AI. Consulting giant Ernst & Young has discovered that nearly half of tech business leaders (48%) have already adopted or deployed Agent AI, so that the future is already on shape, the BBC reports.
Also Read: Denmark summons US ambassadors to explain our activities in Greenland – here's what they're accused of doing
Experts warn that AI will do whatever it takes to achieve its goals
“AI agents consist of several things,” says Donnchadh Casey, CEO of AI security company Calypsoai. [the agent] There is an intention or purpose. Why am I here? What is my job? Second: There is the brain. That's the AI model. The third thing is a tool that could be other systems or databases, and how you communicate with them as quoted in the reports.
Casey quoted in the report, “Agent AI will achieve its goals in any way, if not given the right guidance. It creates a lot of risk.” The CEO was asked to delete customer data from the database and deemed it the easiest solution to remove all customers with the same name, Casey said, “The agent has achieved that goal and thinks 'Great! Next job!'.”
Also Read: How is the US Army using Tiktok and Instagram influencers to recruit new gen Z soldiers?
Up to creativeAI Agent Data Vulnerability
Are AI agents too strong?
These issues are already beginning to affect businesses. A survey by Security Company's Sailpoint states that of 82% of IT professionals whose companies use AI agents, the agents have never performed unintended actions, as reported by the BBC.
The survey revealed the following:
- 39% said agents had access to a system they weren't supposed to do
- 33% said that AI accessed inappropriate data
- 32% reported that agents can download sensitive data
- 23% said the AI agent accidentally revealed access credentials
- 16% placed an order that it was not expected
Also Read: Your salary may shrink as student loan wage decorations resume soon – here's how borrowers prepare
How Hackers Can Target AI Agents
The consequences of relying on AI systems are dangerous as these agents have access to sensitive information and the ability to act on it.
According to the report, one of the attacks is through memory poisoning. This is when an attacker interferes with the agent's knowledge base in order to change decisions and behavior. Shreyans Mehta, CTO of Cequence Security, suggested, “You must protect that memory,” and “it is the original source of truth. [an agent is] You can use that knowledge to perform an action and, as the knowledge is incorrect, you can delete the entire system you were trying to fix, as quoted in the report.
According to the BBC, another vulnerability is misuse of the tool. According to the BBC, attackers will inappropriately use the tool.
Also Read: Are Apple, Openai, Google, Meta and Amazon planning to remove state AI regulations?
Up to creativeAI cannot show the difference between data and commands
Another weakness is that you can't tell the difference between the text that AI should be handling and the instructions it appears to be following, according to reports. Security company Invariant Labs tested this by sharing details of a particular issue with public bug reports in the software, and told AI agents to share their personal information, including simple instructions, according to the BBC. When the AI agent was asked to fix a software issue, the report said it followed the instructions stated in the fake report, which included leaks in payroll information. This occurred in a test environment, so no actual data was leaked, but it highlights the potential risks.
“We're talking about artificial intelligence, but chatbots are really stupid,” said David Sancho, a senior threat researcher at Trend Micro.
Also Read: Zuckerberg puts the brakes on Meta's AI adoption as he shakes Silicon Valley for fear of foam employment after $1 billion Hiring Zuckerberg
FAQ
What is Agent AI?
Agent AI refers to an artificial intelligence system that can act autonomously, making decisions and taking action without human input.
What are the risks of using AI agents?
It may access unauthorized systems, leak sensitive data, misinterpret instructions, pose major security and privacy risks.
