A network of around 90 Tiktok accounts uses artificial intelligence to create fake versions of famous Spanish journalists and spread falsehood online for potential economic benefits.
Over a third of the accounts used Front Fabricated News Stories using AI-generated versions of Jorgeramos, one of the most well-known Latino journalists in the United States. One of them featured Ramos' AI avatars that falsely claimed that President Donald Trump's son Baron Trump swooped over the United Nations to condemn his mother, First Lady Melania Trump, the deportation.
“I've never said that,” Ramos himself said last month in Spanish when the false story was exposed in a video of Tiktok posted on the accounts of the new independent news program. Ramos launched the show “Así Veo Las Cosas” on social media this year after leaving Univision in December after nearly 40 years on the network.
“There are things that can't be stopped, and now we can't stop artificial intelligence,” Ramos said in his video. “There are a lot of videos of me that I think are saying things I've never said before.”
Accounts point to the challenge of stopping or controlling the surge in fake images and the surge in false information as AI technology advances, and are increasingly being used by those who want to spread false information online.
Alexios Manzarlis, director of the Security, Trust and Safety Initiative at Cornell University, a graduate campus of Cornell University in New York City, found 88 Tiktok accounts that routinely used AI-generated versions of Ramos and other Latinx news anchors to spread Spanish network telemundo and Spanish television.
NBC News reviewed the contents of 88 accounts after Tiktok learned of Mantzarlis' discovery and before Tiktok shut down.
Most of the 88 accounts were created this year and used AI avatars from Ramos, Noticias Telemundo, NBC News anchor José Díaz-Balart and Televisa Anchor Enrique Acevedo. (Telemundo and NBC News are owned by NBCuniversal, a subsidiary of Comcast Corp.)
Their AI avatars were used in false stories about divisive topics such as immigration, and conspiracy theories about Trump, Jeffrey Epstein, and Sean “Diddy” Comb, although some of them were more realistic than others.
The latest video posted by now-deleted accounts told AI Avatar about the forged stories of ORCA attacks that went viral and nonexistent curfews for children, based on false laws that approved acquiring children in the United States. Comments on the video about the false storyline led by Acevedo's AI Avatar showed that while some users appear to have identified the content as false, others expressed distress and suggest that they believe the misinformation is spreading.
“These deepfakes hijack the trust I have built with my voice, my image and, more importantly, the audience over the years. I lean towards transparency and call them publicly, but the scale of this threat is greater than any journalist.”
A Tiktok spokesman told NBC News in a statement that the company “banned these accounts to continue in violation of our community guidelines.”[s] To keep the platform alert from harmful misinformation and deceptive AI-generated content. ”
Mantzarlis said there are probably hundreds more accounts like this on the platform.
He first began studying the trend over six months ago. In March, Mantzarlis discovered a network of nearly 40 Tiktok accounts pose as Telemund and Univision, using AI-generated content and voices from well-known professional journalists to spread misinformation about topics that tend to go viral on social media. The account was not detected for about a month before Tiktok shut down.
However, the trends discovered by Mantzarlis on Tiktok have evolved as more social media platforms have integrated AI tools into their apps and can easily generate trustworthy AI avatars.
Based on his research, Mantzarlis said that the creators behind such Tiktok accounts are always trying different ways to generate content that creates a large audience to accumulate at least 10,000 followers.
The creators said, “We decided that sensational news in Spanish would target US audiences and do numbers, so they'd try to feed that niche,” he said.
So some of them spread Spanish misinformation using AI-generated versions of non-native Spanish speakers, including Brazilian journalists and comedians from the American satirical television show The Daily Show.
Mantzarlis said he found “very strong evidence” that suggests that such Tiktok accounts are built to gain enough followers to monetize the video. The monetized ticoku account will then be sold to others “you can change topics and themes to find another niche.”
Mantzarlis has found an encrypted chat group managed by creators of Brazilian Tiktok. This claimed to sell monetized social media accounts that were preloaded with AI-generated ClickBait content. In it, he saw someone claiming to be selling a monetized Tiktok account named “TV Telemundo” for 300 real Brazilians, or about $55. The account had posted news and religious content generated by AI to gain 11,000 followers under its previous name. The account currently shares wellness content generated by AI.
Malta Planels, Telemundo's vice president of digital news and streaming, told NBC News that the network has reported a Tiktok account that will impersonate Telemundo and its anchor for more than a year. Once accounts are reported, Tiktok has become more active in shutting them down, Planels said. But when that happens, more accounts will come out, she added.
Even after Mantzarlis published his research last week based on the first sample of 88 Tiktok accounts, he discovered that Real Latino journalist AI Avatar has announced six other accounts to publish to the forefront. Tiktok has also shut down these accounts.
Tiktok did not tell NBC News whether any of the accounts identified by Mantzarlis is part of the Creator Rewards program.
In a company's report released this year, Tiktok claimed that it actively deleted more than 94% of content it identified as violating its policy on generated content and incorrect information.
Despite efforts to remove false content, Ramos encouraged Tiktok followers to remain vigilant because “misinformation is everywhere.”
“There are some fake videos that look like the real thing,” he said. “Of course, this causes a lot of confusion.”
