Generative AI is all the rage these days, but its rapid adoption could spell trouble: In a dramatic and somewhat comical series of events, TikTok's Symphony Assistant was used to create AI avatars that recited problematic scripts like Hitler's “Mein Kampf” and Osama bin Laden's “Letters to America.”
TikTok announced the Symphony Avatars last month as part of its “creative AI suite” aimed at helping people create personalized videos without hiring real actors. The feature has been rolling out to TikTok Ads Manager accounts over the past week but was originally meant to be limited to those accounts.
CNN producer and digital content creator John Serlin accessed the AI tool using his personal account and discovered there were no safeguards in place for sensitive content.
Generative AI is being used widely in the tech world, and TikTok is no exception: last month the platform first unveiled Symphony Avatars as part of its “Creative AI Suite,” which allows businesses, brands, and creators to create fully customized ads using generative AI and paid actor likenesses (avatars).
The feature was introduced earlier this week but only for users with TikTok Ads Manager accounts. However, this restriction appears to have been temporarily waived when a CNN reporter used a personal account to access one of the Symphony AI tools, where she found the AI tool to have virtually no guardrails or safeguards.
Oh no! TikTok is in trouble again!
Posts from X (formerly Twitter)Sahlin showed several examples of videos created with Symphony Assistant that featured avatars reciting the problematic scripts mentioned above. To make matters worse, the videos bore no watermark or indication whatsoever that they were AI-generated.
Naturally, CNN reached out to TikTok for comment and received the following statement: “A technical error, which has now been resolved, allowed a small number of users to create content using an internal test version of our tool for a few days. If anyone had attempted to upload the harmful content created by CNN, this content would have been rejected for violating our policies. TikTok is an industry leader in responsible AIGC creation and will continue to test and build safeguards that we will apply to all TikTok products before rolling them out to the public.”
It's unclear whether TikTok accidentally released an “internal test version” of the AI tool or simply forgot to put the proper filters and safeguards mechanisms in place, but TikTok says the issue has now been resolved (it's my understanding that Symphony Assistant has been discontinued, at least for now).
Since AI is not going away anytime soon, we can expect to see more stories like this in the future. What do you think? AI is just a tool, and it is ultimately up to us humans to use it immorally. But we need mechanisms to prevent such accidents, right?
