State and local governments across the United States are leveraging AI for a variety of applications. For example, we translate public meetings into multiple languages in real time to enable broader participation, and we use chatbots to provide services to the public.
AI systems can benefit government agencies and the people they serve, but they can also be harmful if misused. In one famous example, nearly 40,000 Michigan residents were wrongly accused of unemployment insurance fraud based on the state’s AI system with flawed algorithms and insufficient human oversight.
“We believe that many AI systems, while potentially useful, are often unreliable and should be treated as such,” said Suresh Venkatasubramanian of Brown University, co-author of the recent National Academies Rapid Expert Consultation on the Use of AI by State and Local Governments.
He urged state and local leaders to avoid extreme hype about the promise and dangers of AI, and instead adopt a cautious and experimental approach. “We need to embrace the spirit of experimentation and sandbox and understand how they work in specific situations.”
Venkatasubramanian spoke at a National Academies webinar that discussed new publications and other AI-related resources for state and local governments. He was joined by co-authors Nathan McNeese of Clemson University and Leila Doty, a privacy and AI analyst in San Jose, California, as well as session moderator Kate Stoll of the American Association for the Advancement of Science.
Focus on purpose and people
When considering whether to implement AI, McNeese advised state and city officials to start by asking, “What’s the problem?” “What aspects of our organization would we like to strengthen?”
“There is no need to deploy AI unless there is a specific need,” McNeese says. “I don’t want to implement AI just because other people are implementing it.”
This point was won by Venkatasubramanian. “If you have a problem that needs to be solved, think about what people need to solve it,” he said. “Maybe AI can be a part of this, maybe it’s not. Don’t start with the question, ‘How can we bring AI into this?'” That’s when the problem arises. ”
When using AI, McNeese explained that the publication recommends a human-centered approach to AI design, an approach that takes into account people’s needs, desires, and motivations.
Venkatasubramanian said that determining where AI tools are useful and where they are not should involve people with specialized subject matter expertise – employees who provide a valuable service to society. “It’s really, really important to have people with expertise understand the field,” he stressed.
The goal should be to create complementary human-AI teams, where humans delegate certain aspects of their jobs to AI, allowing humans to focus on what they’re good at, McNeese said. “We’re not trying to use AI to replace humans.”
AI implementation strategy
McNeese outlined the publication’s strategies for states and cities to plan and implement AI. “Incorporating AI into an organization can be daunting,” he says. “There are a lot of things that have to be considered early in the process.”
After identifying a problem that AI could help solve, he said, agencies should analyze whether implementing AI is feasible and how it will impact existing workflows. Next, agencies must decide what type of AI to use and whether to develop it in-house or use an external vendor.
McNeese noted that the National Institute of Standards and Technology has an AI Risk Management Framework that provides guidance and urged organizations to consider the risks associated with the types of AI they plan to use.
Rather than implementing AI everywhere at once, organizations should pilot AI within subsections, McNeese said. We also need to develop metrics to assess the extent to which AI is contributing to solving the problems to which it is applied. And leaders should ask employees for feedback.
“We want people to have a voice,” McNeese said. “We want people to give us insights, but we also want to adopt, improve, or abandon the use of AI based on that evaluation and feedback.”
Venkatasubramanian said government agencies can seek help in implementing AI through partnerships with local experts, civil society organizations, and universities. “People at universities are very interested in working with state and local governments because these show real-world use cases in real-world environments,” he said, noting that this sheds light on how AI innovation should proceed.
Public Sector AI “Community of Practice”
Another source of assistance available to government agencies when implementing AI is the GovAI Coalition, which was established in November 2023 by the City of San Jose. Leila Doty explained its history and efforts.
The GovAI Coalition was formed after the city of San Jose began rolling out an AI policy and governance framework in early 2023 but struggled to get transparency from some AI vendors about their systems, the data used to train them, and accuracy and other performance metrics. Doty said conversations with colleagues in government agencies across the country revealed that they, too, were facing this problem, and were able to address the broader question of how to provide responsible governance around AI.
“We felt it made a lot of sense to collaborate, collaborate with our peers, and create standards in this space on how local and state government agencies can do AI governance in a responsible way,” she said. Agencies have used their collective strength to demand greater accountability from AI vendors and created a large suite of AI governance resources, including templates for policies and vendor agreements.
“The coalition has really expanded its reach and is now a broader community of AI-centered practices in the public sector,” Doty said, adding that more than 850 agencies (local, county, state, and some federal) across the country are now members.
He welcomed the National Academies’ new publication and noted that its strategy is consistent with the Coalition’s efforts. “I think [they] It will be extremely useful for state and local governments considering implementing AI. ”
Watch the webinar or Read our quick expert consultation .
