Anthropic’s feud with the Department of Defense is more than a contract dispute. This portends a looming power struggle between elected governments and increasingly powerful AI companies.
CEO Dario Amodei said the Pentagon has refused to allow Anthropic’s AI to be used for “any lawful purpose,” raising concerns that it could enable domestic surveillance or autonomous weapons. Jessica Tillipman, a government contracts expert at George Washington Law School, said such a stance could prohibit certain military uses even if the government deems them legal.
This is an example of a potential future where AI CEOs, rather than democratically elected leaders, decide what is acceptable.
Eliezer Yudkowsky, an AI collaboration researcher, warned politicians during the controversy: “It’s not crazy for Silicon Valley executives to think that maybe they could be in charge instead of you.” “If they could actually control superintelligence, they would throw you out like used toilet paper.”
There is a concern here. In a democracy, voters can remove elected officials who make wrong decisions. But when CEOs make harmful choices, especially choices that increase revenue or profits, the public has little recourse. Shareholders, not voters, decide whether a CEO stays in office.
No matter what Amodei or other AI leaders say, companies exist to make profits. Investors poured billions of dollars into Anthropic in hopes of huge future profits. Lofty rhetoric about safety and ethics may be sincere, but it also helps you win customers and recruit talent. “Dario is a better marketer than people give him credit for,” quipped Aakash Gupta, a former product manager.
Anthropic’s own decisions reflect these tensions. In late February, the company backed away from previous promises to pause scaling or delay model deployment if safety measures lagged behind capacity. It’s a surprising change for a company whose CEO has frequently warned about the risk of AI going out of control.
Amodei also shifted its focus to investing in the Middle East when large-scale financing was needed. And this year, Bloomberg reported that Anthropic had applied to compete for a $100 million Pentagon award to design autonomous drone swarm technology.
The pattern is not unique. When Google acquired DeepMind, co-founder Demis Hassabis insisted that the company’s AI could not be used for military purposes. Google agreed, and the deal was done. However, Google has updated its AI Principles by early 2025, removing the pledge to prohibit the use of weapons and surveillance.
Early AI startup Clarify went through a similar process, according to former employee Michal Wolski. “We took the same stance with Clarify in 2014, but then it was time to fully repay the funds raised.”
The result of this inevitable setback: In July, Google, OpenAI, Anthropic, and xAI each signed Department of Defense contracts worth up to $200 million. So if you’re using Gemini, ChatGPT, Claude, or Grok as a chatbot, you’re already supporting an AI company that helps America fight wars.
Technology blogger Ben Thompson argues that the right answer is new laws and responsible oversight, not that he supports letting unelected executives decide how AI is used. It’s “a path toward an even more tyrannical future,” he wrote this week.
Democracy is tricky. But it is better to entrust the levers of power to the CEO, who has the primary duty to shareholders and whose tools may soon become the most important ever built.
An Anthrop spokesperson did not respond to a request for comment Wednesday morning.
Sign up for BI’s Tech Memo newsletter here. Please contact us by email. abarr@businessinsider.com.
