How government AI use hurts democracy

Many countries are investigating how artificial intelligence can help everything from tax processing to welfare benefits decisions. However, research shows that citizens are less enthusiastic than governments, which could create a real risk of democracy.

“To focus solely on short-term efficiency and shiny technologies will trigger public repulsion and contribute to a long-term decline in democratic trust and legitimacy,” says Alexander Utzke at Ludwig Maximilian University in Germany.

Wuttke and his colleagues asked about 1,200 people in the UK to share feelings about the government's actions that humans or AI handled the tasks. These hypothetical scenarios included processing the tax return, approving or denying welfare applications, and conducting a risk assessment of whether the defendant was eligible for bail.

Some people have only been told about how AI can improve government efficiency, while others have also learned about both AI-related benefits and risks. These risks included difficulty understanding how AI decisions are made, increased government reliance on AI that becomes reversible over time, and the lack of a clear path for citizens to compete and revise AI decisions.

When people realized AI-related risks, they reported both a significant reduction in trust in the government and a sense of losing control. For example, the percentage of participants reporting losses in government democratic control increased from 45% to over 81% in a scenario in which governments became increasingly dependent on AI to handle specific tasks.

The proportion of people who lower government AI also increased sharply after participants learned about risk. It increased from less than 20% of the baseline scenario to over 65% in any scenario we learned about both the benefits and risks of government using AI.

Despite these results, the Democratic government can use AI in a responsible way that maintains citizens' trust, says Hannah Key de la Valley of the Center for Democracy Technology in Washington, DC. But she says there have been few AI success stories in the government so far. Meanwhile, there have already been “several cases of failure.” The interests in these cases can be very high.

For example, tens of thousands of people have been mistakenly charged for fraud due to state efforts to automate the processing of US public interest claims. Some of those people had to file for bankruptcy or lose their homes. “Government mistakes have a huge, long-term impact,” says Quay-de La Vallee.