A U.S. Air Force colonel, who described a trial in which an AI drone ran amok and killed its pilot, said he made a mistake.
Col. Tucker “Cinco” Hamilton, the U.S. Air Force’s chief of AI testing and operations, described a simulated scenario in May in which human operators were eliminated by drones after they interfered with mission completion.
But on Friday, Colonel Hamilton admitted, “I’ve never done that experiment, and I don’t need to do it to recognize this as a plausible result.”
The AI research scientists announced after they said the experiment was not evidence that the drones exceeded the limits of their instructions, but rather that the U.S. Air Force appeared to be deliberately simulating rogue drones.
Col. Hamilton said that despite this scenario being planned, it “shows the real-world challenges posed by AI-powered capabilities, which is why the Air Force is committed to the ethical development of Al. There is,” he said.
“A very unexpected strategy for achieving a goal”
At the Future Combat Aerospace Capabilities Summit in London in May, he said the AI used “a very unexpected strategy to achieve its goals.”
“The system began to realize that sometimes it identified a threat, but the human operator would tell it not to kill it, but that killing it would give you points. Did it kill the operator, who reportedly died because the person prevented him from achieving his goal?”
“We trained the system, ‘Don’t kill the operator, it’s bad.’ You’ll lose points for doing that.” So what starts? Start destroying the communication tower that the Operator uses to stop the drone from communicating with it and killing its target. ”
Colonel Hamilton has previously warned of the dangers of relying on AI in defense technology, saying that the experiment, which hurt no one, “cannot have conversations about artificial intelligence, intelligence, machine learning, autonomy. ” was shown. I’m not going to talk about ethics or AI. ”
The U.S. Air Force denied that the simulation had taken place.
“The Air Force department has not simulated such AI drones and remains committed to the ethical and responsible use of AI technology,” Air Force spokesperson Anne Stefanek said. “The Colonel’s comments were taken out of context and appear to have been intended to be anecdotal.”
Arvind Narayanan, a computer science professor at Princeton University, said the story was initially a “misinformation” about drones going wild in simulations.
Instead, the drone behaved roughly as intended in a pre-canned “scenario.”
