It’s not entirely out of character for comedian-turned-podcaster Joe Rogan to endorse a “libido-boosting” coffee brand for men.
But when a recent video circulating on TikTok showed Logan and his guest Andrew Huberman peddling coffee, some keen-eyed viewers, including Dr. I was shocked.
“Yeah, it’s fake.” Dr. Huberman wrote on twitter After seeing the ad, it seems like you’re praising coffee’s testosterone-boosting potential, but that’s actually not the case.
The ad is one of a growing number of fake videos on social media created using artificial intelligence-powered technology. Experts say Logan’s voice appears to have been synthesized using an AI tool that mimics the voices of celebrities. Dr. Huberman’s comments are taken from an unrelated interview.
Creating realistic fake videos, often called deepfakes, once required sophisticated software to match one person’s face to another. But now many of the tools for creating them are available to the everyday consumer. Even smartphone apps often cost little or no money.
The newly altered videos, so far mostly the work of meme creators and marketers, have gone viral on social media sites such as TikTok and Twitter. The content they create, sometimes called cheapfakes by researchers, works by duplicating the voices of celebrities, changing their mouth movements to match another voice, and writing compelling dialogue. .
Video and the accessible technology behind it have made some AI researchers worry about its dangers, raising new concerns about whether social media companies are prepared to manage the growing wave of digital fakes. is causing
Disinformation watchdogs are also preparing for a wave of digital fakes that can deceive viewers and make it difficult to determine what is true online.
“The difference is now anyone can do it,” said Britt Paris, an assistant professor of library and information science at Rutgers University who helped coin the term “cheapfake.” “It’s not just for people with advanced computing skills and fairly advanced computing know-how. Instead, it’s a free app.”
For years, a large amount of manipulated content has been circulating on sites such as TikTok. More hand-crafted tricks are usually used, such as careful editing and replacing one audio clip with another. In one of her videos on TikTok, Vice President Kamala Harris appeared to say everyone hospitalized with Covid-19 had been vaccinated. In fact, she said the patient had not been vaccinated.
Grafica, a research firm that studies disinformation, discovered a deepfake of a fictional newscaster distributed by a pro-Chinese bot account late last year.
But some new tools are bringing similar technology to everyday Internet users, giving comedians and partisans the opportunity to create compelling impersonations.
Last month, a fake video went viral showing President Biden declaring a national draft for war between Russia and Ukraine. The video was produced by the team behind Human Events Daily, a podcast and livestream run by right-wing influencer Jack Posobiek, known for spreading conspiracy theories.
In describing the video, Poszowiec said his team used AI technology to create the video. Conservative account Patriot used the Breaking label in his tweet about the video from Oasis, without indicating that the video was fake. The tweet has been viewed over 8 million times.
Many of the video clips featuring synthesized speech appeared to use technology from ElevenLabs, an American startup co-founded by former Google engineers. In his November, the company announced a speech duplication tool that can be trained to duplicate speech in seconds.
ElevenLabs made headlines last month when 4chan, a message board known for its racist and conspiratorial content, used the tool to share hateful messages. As an example, a 4chan user made an audio recording of an anti-Semitic text using a computer-generated voice that mimicked actor Emma Watson. Motherboard previously reported on 4chan’s use of audio technology.
ElevenLabs said on Twitter: introduce new safeguards, limits audio duplication to paid accounts, and offers new AI detection tools. However, a 4chan user said he would create his own version of the voice cloning technology using open source code and post a demo similar to the voice made by ElevenLabs.
“You need your own custom AI with the power to create,” wrote an anonymous 4chan user in a post about the project.
An ElevenLabs spokesperson said in an email that the company is looking to work with other AI developers to create a universal detection system that can be adopted across the industry.
Videos with duplicate voices created with ElevenLabs tools or similar techniques have gone viral in recent weeks. One posted on Twitter by the site’s owner, Elon Musk, showed a profane, false conversation between Logan, Musk, and Canadian men’s rights activist Jordan Peterson. . In another video posted on YouTube, Logan appeared to be interviewing a disguised version of Canadian Prime Minister Justin Trudeau about his political scandal.
Peterson tweeted about the fake video in which his voice appears, saying, “The production of such fakes should be a crime with a 10-year prison term.” “This technology is incredibly dangerous.”
A YouTube spokesperson said in a statement that the Logan and Trudeau videos “provide sufficient context” and do not violate the platform’s policies. (The creator described it as a “fake video”.
Experts who study deepfake technology have suggested that the fake ads featuring Logan and Dr. Huberman were likely created using a voice cloning program, but the exact tools used have not been confirmed. was not clear. Mr. Logan’s audio was spliced into an actual interview with Dr. Huberman discussing testosterone.
The results are not perfect. Rogan’s clip is taken from another interview with his professional pool player, Fedor Gorst, posted in December. Mr. Logan’s mouth movements do not match the audio, sometimes making his voice sound unnatural. We weren’t sure if the video would have convinced TikTok users, it got much more attention after being flagged as an impressive fake.
TikTok’s policy prohibits digital counterfeiting that “distorts the truth of an event to mislead users and seriously harms the subject matter of the video, other people, or society.” Some of the videos were removed after The New York Times warned the company.Twitter also removed some of the videos.
A TikTok spokesperson said the company “combines technology with human moderation to detect and remove manipulated videos,” but didn’t provide details on how it does it.
Logan and the company featured in the fake ad didn’t respond to requests for comment.
Many social media companies, including Meta and Twitch, ban deceptive deepfakes and manipulated videos. Meta, who owns Facebook and Instagram, has run a competition to develop a program that can identify deepfakes in 2021, resulting in his one tool that can detect deepfakes 83% of the time. .
Federal regulators have been slow to respond. One federal law that went into effect in 2019 requires reporting on the weaponization of deepfakes by foreigners, requires government agencies to notify Congress if deepfakes target U.S. elections, and bans deepfakes. Created an award to encourage research into tools that can detect it.
Ravit Dotan, a postdoctoral fellow who runs the Collaborative AI Responsibility Lab at the University of Pittsburgh, said: “By then, the damage could be too much. We have elections here in America, and that’s going to be a problem.”
