TikTok accidentally posted a link to an in-house version of a new AI digital avatar tool that had no guardrails, allowing users to create videos saying anything. The glitch was first spotted by CNN, which allowed the outlet to generate videos that included phrases like Hitler quotes and messages to drink bleach. TikTok has since removed this version of the tool, but the version TikTok intended to release remains available.
TikTok's Symphony Digital Avatars, which launched earlier this week, allows companies to create ads using the likenesses of paid actors. It also uses AI-powered voice-overs, allowing advertisers to input scripts and have their avatars say whatever they want, within TikTok's guidelines. Only users with a TikTok Ads Manager account can access the tool, but the version CNN found allowed anyone with a personal account to try it out.
In a statement The VergeTikTok spokeswoman Laura Perez said the app had resolved a “technical error” that allowed “a very small number of users to create content using an internal test version of our tools for several days.”
Once CNN discovered the internal tool, it allowed the network to create videos reading Osama bin Laden's “Letter to America,” as well as white supremacist slogans and calls for people to vote on the wrong day. The CNN-created videos do not include a watermark indicating that they were AI-generated, whereas the official version of TikTok's Symphony digital avatar does.
CNN didn't post the video to TikTok, but if it had, the content “would have been rejected for violating our policies,” Perez said. TikTok has since removed the tool, but the question is whether people will find other ways to abuse the digital avatar creators and whether TikTok is prepared to act on them.
