As AI becomes better able to identify software vulnerabilities, experts are increasingly warning of potential disaster scenarios, so-called “Vulnpocalypses.” Hackers could rapidly ramp up their attacks using AI technology designed to identify holes in cyber defenses, security researchers have warned. This week, that scenario started to feel less theoretical.
Subscribe to read this story without ads
Get unlimited access to ad-free articles and exclusive content.
AI giant Anthropic has announced that it will withhold its latest model, Mythos Preview, from public release, citing the potential for serious damage in the wrong hands due to its unprecedented vulnerability discovery capabilities. Instead, the company is sharing its model with a limited group of tech giants and partners to strengthen its defenses.
Concerns extend to the highest levels of government. Following Anthropic’s Mythos Preview announcement, Treasury Secretary Scott Bessent convened a meeting with major financial institutions this week to discuss the “rapid developments occurring in AI,” a Treasury spokesperson said.
Some theorize that AI could help hackers crash financial systems or shut down hospitals and manufacturing plants. It could help countries like Iran shut down critical infrastructure in the United States. Alternatively, it could be used to cause a large-scale outage affecting travelers and internet users.
“There are far more vulnerabilities than most people would like to admit, and while it was already difficult to fix them all, they are now much easier to exploit by a much wider range of potential adversaries,” said Casey Ellis, founder of Bugcrowd, a platform for cybersecurity researchers that tracks vulnerabilities. “AI gives a lot more people the tools they can use to do this.”
Hackers often figure out how to exploit software flaws to break into systems, leading to endless back-and-forths where attackers look for new opportunities and defenders try to update the code to block them. Some AI models, especially those with coding capabilities comparable to or better than humans, have proven to be very good at finding these vulnerabilities quickly.
Concerns about AI’s ability to give hackers a superweapon to overwhelm cybersecurity defenses reached a new high this week when Anthropic announced it would not yet make Mythos publicly available.
But regardless of whether Mythos lives up to its hype, industry experts largely agree that a period of reckoning will likely come soon, when hackers will leverage AI to gain an ever-greater advantage over their victims.
“Defenders have to be right all the time, but attackers have to be right only once,” Ellis said.
Logan Graham, head of offensive cyber research at Anthropic, said he expects the company’s competitors, including those in China, to release models with comparable hacking capabilities in the coming months and years, even if Mythos is not released.
“In six to 12 months, we should be planning for a world where capabilities like this are widely distributed and widely available, not just to businesses in the United States,” Graham told NBC News.
“If you step back, it’s a pretty crazy time frame. Usually it takes years to prepare for something like this,” he said.
Graham said Mythos is good at not just discovering vulnerabilities, but also chaining vulnerabilities together to create complex exploits that become devastating hacking tools.
Katie Moussouris, CEO and co-founder of Luta Security, a company that connects vulnerability researchers and software developers, said she expected a similar scenario to what would happen if a major cloud provider went offline with a glitch and took away a significant portion of the internet.
“We’re definitely going to start seeing large-scale outages that have downstream effects on other industries, just like the airline industry suffered with the CrowdStrike incident. If Cloudflare goes down, if Amazon Web Services goes down, a lot of other sectors will suffer,” she said.
Cynthia Kaiser, a former FBI senior cyber official and senior vice president at Halcyon, which works to prevent ransomware attacks, said she is concerned about how AI will help ordinary hackers whose only limitation to attacking hospitals and demanding ransoms is the fact that they lack the skills.
“Wannabes, an undercurrent of people who didn’t have the ability to conduct these operations just a year ago, now have some of the most powerful tools in human history,” she told NBC News. “Healthcare and critical manufacturing industries were the most targeted by ransomware attacks last year, and I think this pattern will continue. They will target sectors with little tolerance for downtime.”
AI could also have a significant impact on cyberwarfare and attacks on U.S. critical infrastructure by giving hackers an advantage for simple destruction.
Since the beginning of the U.S.-Iranian war, Iranian hackers have targeted multiple U.S. targets and repeatedly overstated their capabilities. They only recorded one major, destructive public attack against Stryker, a medical technology company in Michigan.
Federal agencies announced this week that Iran has had some success in hacking critical infrastructure companies, including water and wastewater services and the energy sector, with the intent to cause chaos. It is unclear whether the attack was serious and the identity of the victim has not been released.
But AI could make that task easier. While some industrial control systems have important cyber defense capabilities, others (for example, some water treatment plants in sparsely populated areas of the country) do not. Such systems are notoriously difficult for hackers because they rely on less well-known systems.
Jason Healy, a senior researcher at Columbia University who specializes in cyber conflict, said Iran has so far been unable to carry out sophisticated cyberattacks against the United States, but AI could make them more viable.
“Instead of raising a generation of hackers who understand water utilities, we need to enable AI to understand these systems and automate the infiltration process,” he said.
Bryson Vaught, founder of Scythe, a platform that helps industrial systems anticipate potential cyberattacks, said critical infrastructure is often cut off from the internet, making true doomsday scenarios unlikely.
“All this does not immediately lead to everyone starting to die, like in Hollywood movies, for example,” he said.
But it’s entirely possible that a persistent hacker with the right access could continue to attack systems such as water treatment plants, temporarily shutting them down until they regain control, he said.
“If it continues to be compromised, at some point it’s going to have to work to actually produce water,” he says.
