California government agencies are fully moving forward with generative artificial intelligence tools to improve government efficiency through AI following Gavin Newsom's 2023 executive order.
One development recently promoted by the governor is a California Department of Forestry and Fire Prevention chatbot, tasked with coordinating the state's wildfire response.
According to a May release from Newsom's Office, Cal Fire says Californians aim to improve access to “critical fire prevention resources and near-real-time emergency information,” according to Cal Fire. However, Calmatters found that they were unable to accurately explain the containment of a particular wildfire, and did not provide information on the list of evacuation items and could not communicate the evacuation order to users.
Newsom has announced that AI applications for traffic, residential and customer service will be implemented in the coming months and years. However, the issue of Cal Fire chatbots raises questions about whether agencies follow best practices.
“The evaluation is not an afterthought,” said Daniel Ho, a law professor at Stanford University, that the study focused on government use of AI. “When piloting and deploying such a system, it should be part of standard expectations.”
The chatbot generates answers using the Cal Fire website and its agent ReadyForwildFire.org. You can communicate to users about topics such as active wildfires, agents, fire preparation tips, and Cal Fire programs. Built by South Carolina-based Citibot, it sells AI-powered chatbots for local government agencies across the country. Procurement records show that Cal Fire will host the tool until at least 2027.
“It really started with the intention and goal of having a better informed public about Cal Fire,” said Issac Sanchez, assistant director of communications at the agency.
When Calmatters asked a bot on Cal Fire bot about whether they are currently active or about basic information about their agency, they gave an accurate answer. However, for other information, Calmatters discovered that if the wording of the query changes slightly, the chatbot can give a different answer, even if the meaning of the question remains the same.
For example, an important way for Californians to prepare for the fire season is to assemble emergency supplies bags if they need to evacuate. “What do I need for an evacuation kit?” I returned a specific list of items from a Cal Fire chatbot. Instead, variations of the question, including “Go Bag”, “Wildfire Ready Kit”, and “Fire Preportness Kit”, returned either a prompt to visit Cal Fire's “Wildfire for Wildfire for Wildfire” site that has that information, or a prompt that returned a message saying “I don't know about the specific item you have” and a Wildfire site link. Two of these terms exist on sites that the chatbot refers to.
Additionally, chatbots did not generate incorrect answers in any of the Calmatters they created, but they don't always get the latest updates.
When asked if it included a ranch fire, a 4,293-acre fire in San Bernardino County, the chatbot said the “latest” update as of June 10th included 50% of the fire. When Calmations queried the chatbot, the information was dated for six days. Fires had previously included 85% of the fires.
Similarly, when asked about current job openings at the agency, the chatbot said there was nothing. A search on the state work site showed two positions that CAL Fire at the time would accept applications.
Mila Gascó-Hernandez is the research director at the University of Albany's Government Technology Center, studying how public institutions use AI-powered chatbots. The two key factors she uses to evaluate such chatbots are the accuracy of the information they provide and how consistently they answer the same question, even if the questions are asked in different ways.
“If the fire comes and you need to know how to respond to it, then you need both precision and consistency in your answer,” she said. “You don't think about 'What is a good way to ask a chatbot?' ”
Currently, the chatbot is unable to provide information regarding evacuation orders related to the fire. When asked who would issue evacuation orders, it said that it sometimes correctly stated law enforcement, but also didn't know that. Sanchez of Cal Fire said it would be reasonable to expect chatbots to answer questions about evacuation.
Without an evacuation order for a particular fire, he said, “The answer must be, 'There appears to be no evacuation related to this incident.' ”
Sanchez said he and his team of about four of his tested the chatbot before they went out by submitting questions they hoped they would ask the public. Cal Fire is now improving bot answers by examining queries that people create and correctly surface the answers that chatbots need.
When Calmatters asked the bot “What will you help?” he replied in early May, “I'm sorry, but I don't have an answer to that question right now,” and asked if Calmatters was asking if they were asking about information about the Cal Fire site. By mid-June, the response was “We can now provide answers to questions related to the information on this page, including current fires, Cal Fire job classification, exam requirements, and details on various Cal Fire programs.
“A big message we want to convey,” Sanchez said.
However, experts said the process of kicking tires with chatbots should happen long before sourcing begins.
According to Stanford's Ho, the priority process has a clear benchmark to evaluate the tool by establishing criteria for how the chatbot is run before the vendor is selected. Ideally, these benchmarks will be created by independent third parties. Also, a benefit and risk assessment is required before the chatbot is released.
And in the best case scenario, the public will be involved before launch, said Gasco Hernandez of Albany. Agents interested in using chatbots should identify questions that the public is likely to ask AI tools in advance. These ensure that your agency provides and refines your chatbot by ensuring public pilot members have the information they ask for their systems.
“These user engagement and user experiences are so important that citizens end up using Chabot,” she said.
That's what this article was like Originally published on Calmatters Reissued under Creative Commons Attribution-NononCommercial-noderivatives license.
