TikTok briefly allowed users to create controversial AI-generated videos.

summary

• TikTok’s Symphony Avatars feature allows businesses and brands to create fully customized ads using only generative AI.
• CNN reporters found that the related Symphony Assistant feature had no guardrails or safeguards whatsoever, and the videos produced in this way were not watermarked.
• TikTok has since resolved the issue, dismissing it as a technical error, and reassuring users that its strict content policies ensure harmful AI videos never make it onto the platform.

Generative AI is widely used in the tech industry, and TikTok is no exception. The platform first unveiled Symphony Avatars last month as part of its “Creative AI Suite,” which allows businesses, brands, and creators to create fully customized ads using generative AI and paid actor likenesses (avatars). The feature was rolled out earlier this week, but only to users with TikTok Ads Manager accounts. However, this restriction appeared to be temporarily waived when a CNN reporter used his personal account to access one of the Symphony AI tools. As a result, the reporter found that the AI ​​tool had virtually no guardrails or safeguards.

The tool in question is Symphony Assistantwas discovered by CNN technology reporter John Serlin, who reportedly had access to it through his regular TikTok account. But to Serlin's surprise, the generative AI feature allows you to create compelling videos on virtually any topic using one of a host of actors. All the reporter had to do was choose an avatar and input a script of his choice, as shown below.

Sahlin went on to reveal several examples of videos that had been created with Symphony Assistant on X/Twitter, including a full recitation of Osama bin Laden's “Letter to America” ​​using the tool (via The Verge).To make matters worse, these videos did not include a watermark indicating they were AI-generated, meaning that unsuspecting users could take such videos at face value if they were published on the TikTok platform.

TikTok says it has resolved the issue

When CNN reached out to TikTok about these videos, a company spokesperson called it a “technical error,” adding that it has since been fixed. The spokesperson also clarified that strict content policies would have prevented such videos from being uploaded to the platform. Below is TikTok's full statement to CNN:

“A technical error, which has now been resolved, allowed a small number of users to create content using an internal test version of our tool for a few days. If anyone had attempted to upload the harmful content created by CNN, this content would have been rejected for violating our policies. TikTok is an industry leader in responsible AIGC creation and will continue to test and build safeguards that we will apply to all TikTok products before rolling them out to the public.”

Apparently, TikTok accidentally pushed out an “internal test version” of the AI ​​tool to all users instead of a select few. Luckily, CNN reported the issue in time, forcing TikTok to hit the pause button. The ByteDance-owned company didn't say whether Symphony Assistant would be back anytime soon, but the company's statement makes it clear that it won't stop testing new AI features.

This isn't TikTok's first foray into AI, and it probably won't be its last, but given that TikTok is one of the most used social media apps in the U.S., regulators are unlikely to be too happy about this, especially given the ongoing efforts to ban the app.