Stay informed with free updates
Just sign up for artificial intelligence myFT Digest — delivered straight to your inbox.
The writer is Director of International Policy, Cyber Policy Center, Stanford University, Special Adviser, European Commission
Western governments are racing to establish AI safety institutes. The UK, US, Japan, and Canada have all announced such efforts, and the US Department of Homeland Security just last week added an AI Security Commission to the effort. It is noteworthy that none of these agencies controls the military use of AI, given the heavy emphasis on safety. Meanwhile, the potential for clear AI safety risks has already been demonstrated on the modern battlefield.
According to a recent investigation by Israeli magazine +972, the Israel Defense Forces used an AI-enabled program called Lavender to flag targets for drone attacks. The system combines data and sources to identify armed suspects. The plan reportedly identified tens of thousands of targets, and the bombs dropped in Gaza caused excessive collateral deaths and damage. The IDF denies some aspects of the report.
Venture capitalists are driving the “deftech” (defense technology) market. Technology companies are keen to join this latest boom and are eager to tout the benefits of AI on the battlefield. Microsoft has reportedly sold its generative AI tool, Dalle-E, to the US military, while controversial facial recognition company Clearview AI hopes to help Ukraine use its technology to identify Russian soldiers. I'm proud of what I did. Anduril develops autonomous systems and Shield AI develops AI-powered drones. The companies raised hundreds of millions of dollars in their first round of investment.
But while it's easy to blame private companies for touting AI for war purposes, it's governments that have allowed the “deftech” sector to escape scrutiny. The landmark EU AI law does not apply to AI systems “for military, defense or national security purposes only”. Meanwhile, the White House executive order on AI had important provisions regarding military AI (although the Department of Defense has internal guidelines). For example, many implementations of the executive order “exclude AI used as a component of national security systems.” And Congress has taken no action to regulate military uses of the technology.
As a result, the world's two major democracies will have no new binding rules on what kinds of AI systems their militaries and intelligence agencies can use. Therefore, they lack the moral authority to encourage other countries to put guardrails on the use of AI in their militaries. The recent political declaration on “Responsible Military Use of Artificial Intelligence and Autonomy” supported by many countries is just that: a declaration.
We need to ask ourselves how meaningful the political debate about AI safety is if it does not cover military uses of the technology. Despite the lack of evidence that AI-powered weapons can comply with international law of distinction and proportionality, they are being sold around the world. The lines between civilian and military use are becoming blurred, as some technologies are dual-use.
The decision not to regulate military AI comes at a human cost. These systems are often given unwarranted confidence in military contexts because they are mistakenly assumed to be fair, even though they are systematically inaccurate. While AI can certainly help make military decisions faster, it can also be error-prone and fundamentally non-compliant with international humanitarian law. Controlling human operations is essential to holding actors legally accountable.
The United Nations has sought to fill that void. Secretary-General Antonio Guterres first called for a ban on autonomous weapons in 2018, calling them “morally repugnant.” More than 100 countries have expressed interest in negotiating and adopting new international law to prohibit and limit autonomous weapons systems. However, Russia, the United States, Britain and Israel opposed the binding proposal, and negotiations broke down.
If states do not act to protect civilians from the military use of AI, we need to strengthen the rules-based international system. The UN Secretary-General's High-Level Advisory Group on AI (of which I am a member) would be one of several bodies suitable to recommend banning dangerous uses of military AI, but compliance with the rules Political leadership remains essential to ensure that
In this new era of war, it is important that human rights standards and the laws of armed conflict continue to protect civilians. We cannot continue to use AI in a chaotic manner on the battlefield.
