Top AI (AI) companies say the technology was “weaponed” by hackers and carried out sophisticated cyberattacks.
The humanity who makes Chatbot Claude says that the tool was used by hackers “to commit massive theft and fear torment of personal data.”
The company said AI was used to help create code to execute cyberattacks, but in another case, North Korean fraudsters used Claude to take remote jobs at top US companies.
Humanity says it was able to disrupt threat actors, improving its detection tools and reporting the incident to authorities.
Using AI to assist in writing code has become more popular as technology becomes more capable and accessible.
Humanity says it has detected cases of so-called “vibe hacking.” In this case, AI was used to create code that could be hacked into at least 17 different organizations, including government agencies.
The hacker said they “used AI on what we believe is an unprecedented degree.”
They use Claude to “make tactical and strategic decisions, such as how to decide which data to remove and create requests for psychologically targeted tor.”
It even hinted at the victims at the cost of their ransom.
Agent AI is advertised as the next big step in space when technology is run autonomously.
However, these examples illustrate some of the risks poses by powerful tools to potential victims of cybercrime.
Using AI “means that time needed to take advantage of cybersecurity vulnerabilities will rapidly reduce,” said Alina Timofeeva, cybercrime and AI advisor.
“Detection and mitigation measures must be turned into a proactive and preventive rather than reactive after causing harm,” she said.
However, cybercrime is not the only one that uses technology.
Humanity said “North Korean operatives” will use the model to create fake profiles and apply for remote employment at a US Fortune 500 high-tech company.
The use of remote jobs to gain access to corporate systems has been known for some time, but Anthropic says using AI in fraud schemes is a “funny new phase in these employment frauds.”
He said that AI was used to create recruitment applications and when scammers were hired, it was used to help translate messages and write code.
In many cases, North Korean workers are “culturally and technically sealed from the outside world, making it difficult to separate this modest thing,” said Jeff White, co-host of the BBC podcast's Lazaro Heist.
“Agent AI can help them jump over those barriers and allow them to be hired,” he said.
“Their new employers have since violated international sanctions by unconsciously paying North Korea.”
However, he said that AI “is not creating a whole new criminal at the moment” and “many ransomware intrusions have been happening thanks to tricks tested in attempts such as sending phishing emails and looking for software vulnerabilities.”
“Organisations need to understand that AI is a repository of sensitive information that needs protection, just like any other form of storage system,” said Nivedita Murthy, a senior security consultant at cybersecurity firm Black Duck.
