AI-generated audio and video platforms are rapidly improving and proving potentially useful in fields from entertainment to human resources. But its leaders acknowledge there are risks and say they need to be careful in enforcing moderation policies that prevent users from deploying AI to impersonate public figures or commit fraud. .
talk to luck At the Brainstorm AI London conference, senior writer Jeremy Kahn, Victor Riparbelli, CEO of Synthesia, and Mati Staniszewski, CEO of ElementalLabs, talk about making sure their company's voice cloning and video generation technology is put to good use. He said he is still looking for ways to do so.
“As with most new technologies, we immediately consider what could go wrong. In this case, you're right. Bad people could use this to do bad things. There's no question about it,” said Riparbelli, co-founder of the video generation platform.
Impersonating celebrities using AI-generated audio and video has emerged as a controversial topic. In recent months, blatant deepfakes of Taylor Swift and fake robocalls of Joe Biden have been created, leaving observers concerned about how AI-generated content will affect this fall's U.S. presidential election.
OpenAI recently announced that it would be delaying the rollout of its AI voice cloning tool, highlighting the risk of abuse and potential political repercussions. '' said in a statement.
said Staniszewski, co-founder of AI voice cloning company Eleven Labs. luck's Khan said his company is investing in know-your-customer protocols and mandatory disclosures to ensure all content generated on the platform is linked to a specific user's profile. The company is also exploring different ways to make it clear to users what content is AI-generated and what isn't, he said.
“All content generated by Eleven Labs is traceable to a specific account,” Staniszewski said. “One of the things we're advocating for is not just watermarking things that are AI, but watermarking the actual content.”
Lipalberg said Synthesia has protocols in place that require users to verify their identity and provide consent before creating AI-generated videos.
“Today, it’s impossible to go in and shoot a YouTube video and clone someone. [on Synthesia]. That’s how we control it,” Riparbelli said. “We have very strict content controls and have rules about what you can and cannot create.”
A reporter asked about the potential risks of audio deepfakes in relation to London Mayor Sadiq Khan, who was the target of a viral audio clip last November impersonating Sadiq Khan criticizing pro-Palestinian marches. .
Mr Khan told the BBC: “Parliament needs to wake up and realize that if it doesn't act, it will give the mischief-makers an opportunity to become even more emboldened.”
“All content out there should be known as AI-generated, and there should be tools that allow you, as a user, to get that information at your fingertips… so Sadiq Khan could send a message. , we can confirm that this is a genuine message,''' Staniszewski said.
Lipalberg said it will take time for the industry and lawmakers to come to an agreement on how best to use and regulate tools like those offered by his and Staniszewski's companies.
“Like any new technology, it's going to be a year where people figure out what's right and what's wrong,” Lipalberg said.
