Historically, humans were the only humans to make life-or-death decisions regarding the use of force and deadly weapons. However, society is now entering a new era in which machines equipped with artificial intelligence have the ability to autonomously make lethal decisions without involving humans in the decision-making chain. Does the development of autonomous robots for lethal military applications signal that we have entered a brave new world, or a virtual minefield?
According to recent reports, the Pentagon was dissatisfied with the AI company Anthropic because its executives did not agree to the Pentagon’s request that Anthropic rescind certain guidelines for AI models used by the military.1 Anthropic has a long track record of its AI models being used by the military in classified clouds and other intelligence and military applications. While Anthropic clearly had no problem monetizing its AI models for most military applications, the company wanted to limit the military’s use of AI in two different ways.
- mass surveillance. Anthropic argues that AI-enabled mass surveillance poses serious risks to fundamental freedoms. Anthropic did not want to allow its AI models to be used for large-scale domestic surveillance in the United States.
- fully autonomous weapon. Anthropic wanted to limit the use of AI models for fully autonomous weapons that completely remove humans from the decision-making loop and automate targeting with lethal actions.
Let’s consider each of these issues and the serious questions raised.
mass surveillance
As Orwellian as it sounds, we know that governments have the ability to monitor us. For example, video cameras are being installed explosively throughout society. Furthermore, as a result of digitization, much of our personal data is now available on the digital cloud. AI models can sift through this vast world of video and data using facial recognition software and profiling algorithms. In this way, AI computers have the means to collect vast amounts of public data, combined with powerful tools that can be used for mass surveillance of citizens.
Video cameras serve many useful functions, including allowing law enforcement to access video to prevent and solve crimes. Databases are a critical component for many organizations as well, and our personal data, from photos and documents to data consumption patterns, details our identities, habits, travel, spending, and phone calls. While discovering online patterns may be useful for preventing crime or marketing products, these vast amounts of personal data can be misused to invade privacy, commit identity theft, facilitate censorship, or worse.
Recently, we seem to have crossed a breaking point where society may face government surveillance and privacy issues. Antropic balked at the military’s reluctance to accept limits on how AI models can be used. Pentagon officials argued that the military should be bound only by U.S. law, not the ethics policies of private companies.
fully autonomous weapon
The second key area where Anthropic sought restrictions on the military’s use of AI models was the use of autonomous AI military drones to make deadly decisions. Psychology has studied how humans make important decisions under duress, from athletics to law enforcement to the military.2 For example, risk sensitivity theory provides a framework for understanding the complex human cognitive processes that make the difference between life and death.
Can we be confident that AI can make life-or-death decisions while balancing all variables? Research on human decision-making suggests that AI should be trained not just to aim for the highest reward (e.g., more “kills”), but to actively avoid catastrophic outcomes and manage uncertainty with human-like ethical responsibility.
Anthropic didn’t want its AI models to be used to make deadly autonomous decisions without humans in the chain of command for final decision-making. The company proposed that only humans, not AI, should be able to ethically make the fatal decision to “pull the trigger.”
The Department of Defense disagreed with this restriction and Anthropic was released as the Department of Defense’s AI provider. Other companies soon offered their AI models to the Pentagon on their behalf, but debate will continue over whether private companies developing AI models can define their own guardrails and limits on the military’s use of the software. The U.S. Congress has yet to enact laws or even clear rules to guide AI companies and the Department of Defense on the ethical limits of AI in military applications.
Artificial Intelligence Essentials
The debate is complicated by the fact that athletes from other countries may not be restricted by ethical guidelines. In the global race to militarize AI, the rapid invention of military drones and robots makes it increasingly likely that in the future machines will be able to autonomously make targeting and killing decisions. Will the U.S. military abandon its possession of such tools while adversaries develop autonomous, lethal drones unfettered by ethics? Will the US unilaterally commit to limits on the use of AI in lethal military operations?
The consequences of such a decision are alarming. As with many technologies, the genie is probably already out of the bottle. Lethal autonomous military drones have already been developed and deployed on battlefields around the world. The AI may be smart enough to distinguish between enemy and friendly tanks, but the question is whether it should make the decision to pull the trigger.
In the fog of war, mistakes happen and fatal errors can occur. Are we ready for AI-powered robots and drones to decide whether they are friend or foe, who to stop and who to spare? Ultimately, psychology can study how humans make important decisions. But we must decide as a society how to train and manage the decision-making of AI, including how it will be used to screen personal data and whether we will be allowed to equip lethal autonomous drones and robots to make life-or-death decisions.
