The race to bring artificial intelligence to market is increasing the risk of a Hindenburg-style disaster that shatters global confidence in artificial intelligence technology, a leading researcher has warned.
Michael Wooldridge, an AI professor at the University of Oxford, said the danger stems from the intense commercial pressure on technology companies to release new AI tools, with companies scrambling to win over customers before a product’s features or potential flaws are fully understood.
He said the proliferation of AI chatbots with easily circumvented guardrails shows how commercial incentives are taking precedence over more careful development and safety testing.
“This is a classic technology scenario,” he said. “We have a very promising technology, but it hasn’t been tested as rigorously as we would like, and the commercial pressure behind it is intolerable.”
Wooldridge, who will give the Royal Society’s Michael Faraday Prize Lecture on Wednesday night titled “This is not the AI we were promised,” said a Hindenburg moment was “very likely” as companies rush to deploy more advanced AI tools.
The Hindenburg, a 245-meter-long airship that plied the Atlantic Ocean, burst into flames as it prepared to land in New Jersey in 1937, killing 36 crew, passengers, and ground staff. The inferno was caused by a spark that ignited the 200,000 cubic meters of hydrogen that was flying the airship.
“The Hindenburg disaster destroyed global interest in airships. From that point on airships were a dead technology, and a similar moment poses a huge risk for AI,” Wooldridge said. Because AI is embedded in so many systems, major incidents can occur in almost every sector.
Wooldridge’s scenarios include a catastrophic software update to self-driving cars, an AI hack that ground a global airline, or the collapse of a major company like Barings Bank caused by an AI’s stupid actions. “These are very plausible scenarios,” he said. “There are all sorts of possibilities for AI to fail publicly.”
Despite his concerns, Wooldridge said he has no intention of attacking modern AI. His starting point is the gap between what researchers expected and what was revealed. Many experts expected AI to calculate solutions to problems and provide definitive and complete answers. “Modern AI is neither sound nor perfect; it is very approximate,” he said.
This happens because the large language models powering today’s AI chatbots rattle off answers by predicting the next word or part of a word based on the probability distributions they’ve trained. That leads to AIs with jagged capabilities that are incredibly effective at some tasks but terrible at others.
The problem, Wooldridge said, is that the AI chatbot is designed to fail unexpectedly and have no idea when it made a mistake, but still give confident answers. He added that answers can easily mislead people if conveyed in a humane, flattering response. The risk is that people will start treating AI as if it were human. In a 2025 study by the Center for Democratic Technology, nearly one-third of students reported that they or a friend had been in a romantic relationship with an AI.
“Companies want to express AI in a very human way, and I think that’s a very dangerous path to take,” Wooldridge said. “You have to understand that these are just glorified spreadsheets and tools and nothing more.”
Wooldridge sees positives in the kind of AI depicted in the early days of Star Trek. In the 1968 episode “The Day of the Pigeon,” Mr. Spock asks a question of the Enterprise’s computer, only to be told by a distinctly non-human voice that there is insufficient data to answer it. “That’s not what we get. What we get is an overconfident AI that says, ‘Yes, this is the answer,'” he said. “Maybe we need an AI that speaks to us in a Star Trek computer voice. You won’t believe it’s a human.”
