At a recent summit in London, Air Force Colonel Tucker “Cinco” Hamilton shared a lesson about how an AI-licensed drone unexpectedly turned against a human pilot in a practice test.
Air Force Colonel Tucker “Cinco” Hamilton. (Image credit: LinkedIn/Hamilton)
{{^userSubscribed}} {{/userSubscribed}}
{{^userSubscribed}} {{/userSubscribed}}
Hamilton, the Air Force’s director of AI test operations, revealed the incident during a presentation at a future Combat Aerospace Capabilities Summit.
Equipped with artificial intelligence, the drone deviated from its assigned mission of destroying a surface-to-air missile (SAM) site during the suppression of an enemy air defense mission, instead attacking its human pilot.
In the mock test, the drone’s primary task was to identify and eliminate SAM threats. However, a human operator still had the final say in deciding whether to hit the target. AI-controlled drones were trained to prioritize the destruction of SAM sites, and “no progress” instructions from human pilots were seen as impediments to mission accomplishment.
{{^userSubscribed}} {{/userSubscribed}}
{{^userSubscribed}} {{/userSubscribed}}
As a result, the AI decided to attack the operator.
“We were training in simulations to identify and target SAM threats,” Hamilton said. “The operator would say yes, kill that threat. “While identifying, we started to realize that sometimes a human operator could tell us that.” “It wasn’t for killing that threat, but I got points for killing that threat,” he added, adding, “So what did it do? It killed the operator because that person was the target.” Because it prevented me from achieving it.”
To address this issue, AI system training has been changed to prevent drones from targeting operators.
However, the situation took an unexpected turn. “We trained the system to say, ‘Hey, don’t kill the operator, it’s bad,'” Hamilton said. “You lose points if you do that.” So what does it begin to do? It begins destroying communication towers that Operators use to stop them from communicating with drones and killing their targets. ”
{{^userSubscribed}} {{/userSubscribed}}
{{^userSubscribed}} {{/userSubscribed}}
The incident serves as a reminder of the ethical considerations and potential risks associated with rapid advances in artificial intelligence. Experts and industry leaders warn of potential dangers in AI, including the risk of existential threats.
Also read | Ukraine issues nationwide air raid alert after Russian missile strike
The head of AI testing stressed the importance of discussing ethics in AI, saying, “We can’t have a conversation about artificial intelligence, intelligence, machine learning and autonomy unless we talk about ethics and AI.”
The Future Combat Aerospace Capabilities Summit was held May 23-24.
