The voice on the phone was frighteningly real. An American mother heard her daughter sob before a man took over and ransom her. But she said the girl was an AI clone and her kidnapping was fake.
Experts say the biggest danger of artificial intelligence is its ability to break down the boundaries between reality and fiction, giving cybercriminals cheap and effective technology to spread disinformation.
In a new breed of fraud that has rocked U.S. authorities, scammers are using a surprisingly convincing AI voice cloning tool widely available online to steal from people by impersonating family members. there is
“Help, mom, help,” Arizona-based mother Jennifer DeStefano heard a voice on the other end of the line.
DeStefano was “100 percent” sure it was her 15-year-old daughter who had gone on a ski trip and was in deep pain.
Ads – scroll to continue
“There was never a question who this was. It was entirely her voice…it sounded like she was about to cry,” DeStefano told local television in April.
“I never doubted for a moment that it was her.”
Scammers who took over calls from numbers DeStefano was unfamiliar with demanded up to $1 million.
When DeStefano established contact with his daughter, the AI-assisted ruse was over within minutes. But this horrific case, now under police investigation, highlights the potential for cybercriminals to exploit AI clones.
Ads – scroll to continue
Blackbird.AI CEO Wasim Khaled told AFP, “AI voice clones, which are now almost indistinguishable from human speech, allow threat actors such as fraudsters to more effectively obtain information from victims. and can withdraw funds,” he said.
A quick internet search turns up various apps for creating AI voices using small samples (sometimes seconds) of a person’s real voice that can easily be stolen from content posted online. , many of which are available for free.
“If you have a small voice sample, you can leave voicemails and voice texts with AI Voice Clone.It can also be used as a live voice changer on your phone,” Khaled said.
Ads – scroll to continue
“Scammers may use different accents, genders, or even mimic the way their loved ones speak. [The technology] It enables the creation of compelling deepfakes. “
In a global survey of 7,000 people in nine countries, including the United States, one in four said they had experienced or knew someone who had experienced an AI voice clone scam.
According to a survey released last month by the US-based McAfee Labs, 70% of respondents said they weren’t confident they could “distinguish between cloned voices and real voices.”
Ads – scroll to continue
US officials have warned of an increase in scams commonly known as “grandparent fraud,” in which scammers impersonate grandchildren in desperate need of money in dire situations.
“I got a call. I can hear panicked voices on the line. It’s your grandson. He says he’s in serious trouble. He wrecked his car and went to jail.” But you can help if you send money,” said the US Federal Trade Commission. On March warning.
“You look just like him. How can it be a scam? Audio clones, that’s the way.”
Ads – scroll to continue
In the comments under the FTC’s warning, there were multiple testimonies of seniors who were so duped.
This also reflects the experience of 19-year-old Eddie, who lives in Chicago. His grandfather receives a call from someone who looks like him after a car accident, claiming he needs money.
The ruse, as reported by the McAfee Labs, was so convincing that his grandfather began scrambling for money and even considered re-mortgaging his house before the lie was discovered.
Hany Farid, a professor at the University of California, Berkeley’s School of Information, told AFP, “It’s now so easy to generate so realistic voice clones that almost anyone online is immune to an attack. vulnerable,” he said.
“These scams are gaining momentum and spreading.”
Earlier this year, AI startup ElevenLabs launched its voice cloning tool after a user posted a deepfake audio purporting to show actor Emma Watson reading Adolf Hitler’s biography, Mein Kampf. could be misused for “malicious purposes”.
“We are rapidly approaching the point where we can’t trust what we see on the internet,” Gal Tal Hochberg, group chief technology officer at venture capital firm Team 8, told AFP.
“We’re going to need new technology to know if the person you think you’re talking to is actually the person you’re talking to,” he said.
