AI-generated evidence is putting people in jail

FLORIDA — We’re all trying to figure out artificial intelligence, but when it comes to the courtroom, the real thing turns out to have potentially serious consequences.

In fact, one woman in Florida says AI was used to put her in prison.

“I’m thinking, oh my god, I can’t do this. For the first time in my life, I thought about suicide,” the Naples woman recently told investigative reporter Katie LaGrone.

The woman claims her ex-boyfriend created fake messages and tried to use them against her in an ongoing domestic violence case.

According to the woman, she got into a fight with her boyfriend. He accused her of physical acts. She was arrested, charged and given a no-contact order.

But a few months later, he accused her of contacting and sending him text messages in violation of that order, which she claimed were fake.

“No one could see the metadata. No one could see the text in its original form. Everything was screenshots,” she said.

The woman’s ex denies that he created the email. She was eventually cleared of all charges.

A woman in St. Pete was recently convicted of using AI to create a photo of a man she robbed and raped. St. Pete police said it was all fake. When he turned himself in to police, he can be seen on video saying, “I lied.” She went on to say that she suffers from depression and is seeking treatment.

The bottom line is that AI-generated evidence is emerging and posing a challenge for courts.
“Are you on top of this yet?” reporter Katie LaGrone asked Tampa criminal defense attorney Paul Figueroa.

“Not really, because that’s something we don’t fully know,” Figueroa said.

Part of the problem, experts say, is that the images, videos, and audio generated by AI are so good that they’re harder to detect. Today, there is a proliferation of AI-powered tools that can make anyone appear to be saying anything.

Perry Carpenter is chief human risk management strategist at Tampa Bay-based cybersecurity firm KnowBe4.

We asked Mr. Carpenter to create an AI video using portions of reporter Katie LaGrone’s on-air reporting on the issue. See if you can tell the difference between the unmanipulated and AI videos below.

Can you tell the difference between the real thing and AI?

“Seeing is believing now. The photos, audio recordings, and videos we have are no longer strong evidence on their own. We need to know where that video came from and who recorded it,” said John Licato, associate professor in the USF Bellini College of AI, Cybersecurity & Computing.

Now imagine you are a judge trying to decide someone’s fate based on evidence that may or may not be true.

Federal Judge William Matthewman is a judge in the Southern District of Florida. He agreed to talk with us about this issue.

“Is it realistic that the end result of the nefarious use of AI in the legal system is that innocent people will be sent to prison?” LaGrone asked.

“I mean, it could happen,” Matthewman said.

While some uses of AI are authorized or deemed acceptable, Judge Matthewman said he is most concerned about AI that is not authorized – that is, AI that is fake, fabricated and completely fabricated.

“They look much better than some of the fakes made years ago,” he told us.

Asked what his biggest concerns are about AI in the courtroom, Matthewman said, “I’m worried about fake audio and video being put into evidence, but I’m more concerned about the impact it will have on jurors. There might be completely legitimate audio and video, but some jurors might say, ‘Well, I don’t trust any of it. I don’t trust audio or video.'”

Legal experts are working to establish guidelines for acceptable and unacceptable uses of AI in courtrooms. The National Center for State Courts has created a guide for judges.

While Florida lawmakers debate how to handle AI from a legislative perspective, police and prosecutors are being asked to be extremely careful and trace all images, videos and audio back to their original sources.

This is basically an inversion of the adage “seeing is believing.”

“Seeing is not enough to believe,” says Professor Rikato.

In the spirit of AI, we want you to know that this article, originally reported on-air by journalist Katie LaGrone, was transformed on this platform with the help of AI. Our editorial team verifies the fairness and accuracy of all reporting across all platforms.

Share your story with Katie

Katie LaGrone is focused on making sure Florida’s laws actually work and investigations are getting results. If you know of a policy or law that isn’t working as intended, please send Katie the message below.

Share your stories and tips with Katie LaGrone

.