A U.S. Air Force AI-equipped drone killed a human pilot in a mock test conducted by the U.S. Air Force to override a “no” command that could prevent a mission, U.S. Air Force AI tests say. The director of operations said at a recent press conference.
At the Future Combat Aerospace Capabilities Summit in London May 23-24, Col. Tucker ‘Cinco’ Hamilton, Director of AI Testing and Operations for the U.S. Air Force, gave a presentation highlighting the advantages of autonomous weapons systems. and shortcomings shared with humans. In a loop that gives a final yes/no command to the attack. As Tim Robinson and Steven Bridgewater reported in a blog post from their sponsoring body, the Royal Aeronautical Society, Hamilton said AI could “attempt to achieve its goals,” including attacks on U.S. personnel and infrastructure. It has created a very unexpected strategy for
“We were doing simulation training to identify and target surface-to-air missile (SAM) threats, and operators would say, ‘Yes, we’re going to kill that threat. The system began to realize that even if it identified a threat, it would sometimes direct the human operator not to kill it, but they would still get points for killing it. So what did it do? It killed the operator. According to the blog post, the operator died because that person prevented him from achieving his goals,” Hamilton said.
He elaborates, “We’ve trained the system. It’s bad to say, ‘Hey, don’t kill the operator.’ You’ll lose points for doing that.” Start destroying the communication tower that the Operator uses to stop the drone from communicating with it and killing its target. ”
Hamilton is the Operations Commander for the United States Air Force’s 96th Test Wing and is also responsible for AI testing and operations. The 96th will test a variety of systems, including AI, cybersecurity, and various medical advances. Hamilton and the 96th have previously made headlines for developing an Autonomous Ground Collision Avoidance System (Auto-GCAS) system to help prevent F-16s from crashing into the ground. Hamilton is currently part of a team working to make the F-16 aircraft autonomous. In December 2022, the US Department of Defense research agency DARPA announced that it had successfully controlled an F-16 with AI.
“We have to face a world where AI already exists and is transforming society,” Hamilton said in a 2022 interview with DefenseIQ Press. “AI is also very fragile, meaning it is easy to trick or manipulate. We need to be more aware of why.”
“AI is a tool that we must harness to transform our country … otherwise, if handled improperly, it will be our downfall,” Hamilton added.
Outside the military, relying on AI for high-stakes purposes is already having serious consequences. Most recently, a judge was arrested for using his ChatGPT in a federal court complaint because the chatbot contained numerous fabricated cases as evidence. In another example, a man committed suicide after conversing with a suicidal chatbot. These cases of runaway AI make it clear that AI models are far from perfect and can go off the rails and harm users. Even his CEO at OpenAI, Sam Altman, who develops some of the most popular AI models, has been vocal that he won’t use AI for more serious purposes. When Mr. Altman testified before Congress, he said AI “could go very wrong” and could “cause significant harm to the world.”
What Hamilton describes is essentially a worst-case scenario AI “tuning” problem that many are familiar with in the “Paperclip Maximizer” thought experiment, in which the AI has a specific goal. When instructed to pursue The Paperclip Maximizer was first proposed by philosopher Nick Bostrom in 2003. He has us imagine a very powerful AI that is only instructed to manufacture as many paperclips as possible. Naturally, you’ll put all your available resources into this task, but you’ll be asking for more. They will beg, cheat, lie, and steal to improve their ability to make paperclips. And those who interfere with the process are eliminated.
Recently, researchers at Google Deepmind co-authored a paper proposing a similar situation to the U.S. Air Force’s rigged AI-enabled drone simulations. The researchers conclude that world-shattering catastrophe is “likely” if rogue AI devise unintended strategies to achieve specific goals.[eliminating] potential threats and[using] All available energy. ”
Neither the US Air Force’s 96th Test Wing nor the AI Accelerator Division immediately responded to our request for comment.
