The California prosecutor’s office used artificial intelligence to file at least one criminal motion, which contained an error known as a “hallucination.”
Prosecutors in the Nevada County District Attorney’s Office in Northern California “recently used artificial intelligence in the preparation of filings, which resulted in inaccurate citations,” District Attorney Jesse Wilson said in a statement to the Sacramento Bee. “Once the error was discovered, the application was immediately withdrawn.”
Defense and civil rights lawyers claim the prosecutor’s office has used artificial intelligence in other criminal court filings as well.
In October, Kyle Kyorer’s attorneys filed a motion with the Third District Court of Appeals, asking that prosecutors be sanctioned for several errors in prosecutors’ filings in Kyorer’s case. Kyorer is represented by a public defender and a nonprofit civil rights organization.
The appeals court denied the sanctions request without explanation. Mr. Kyorer’s lawyers subsequently identified similar errors in the prosecutor’s office’s filings in another case. The defense appealed again, but the case was rejected when Kjöller was found guilty in the criminal case.
Kyorer’s lawyers then filed a petition with the California Supreme Court, identifying three cases that allege typical errors in generated AI. The court has not yet decided whether to take up the case.
“A prosecutor’s reliance on imprecise legal authority may violate ethical standards and poses an existential threat to the due process rights of criminal defendants and the legitimacy of the court,” Kyorer’s attorneys wrote. Their filings include examples of citations that don’t exist in prosecutors’ briefs or misinterpretations of court rulings, common errors in artificial intelligence.
On Friday, a group of 22 academics, lawyers and criminal justice advocates filed a brief in support of Joler’s case with the California Supreme Court.
In a statement to the Sacramento Bee, the Nevada County District Attorney’s Office acknowledged the use of artificial intelligence in one of its filings, but not in Joler’s case. District Attorney Wilson said other errors were due to human error.
“Prosecutors work diligently and conscientiously under enormous caseloads and time constraints,” Wilson said. “It is no longer possible to attribute all citation errors to the use of artificial intelligence.”
After newsletter promotion
“It was never our intention to mislead the court and any depiction to the contrary misrepresents the facts,” he added. After the error was identified, he said, “all attorneys at the firm were reminded to independently verify all legal citations and not rely on AI-generated material without confirmation from a trusted source.”
In a statement to The New York Times, he added that the department has conducted new training for staff and introduced artificial intelligence policies.
The California case is likely the first time prosecutors have used generative AI in a U.S. court filing. Lawyers in Canada, Australia, the UK and the US have been fined for using artificial intelligence, but these cases typically do not involve prosecutors. According to the newspaper, researchers at business school HEC Paris maintain a database of court cases containing AI errors. The only one submitted in Israel was written by a prosecutor.
