AI videos are flooding social media. no one was ready

A video posted to TikTok in October shows a woman being interviewed by a television reporter about food stamps. The women weren't real. The conversation never happened. This video was generated by artificial intelligence.

Nevertheless, people seemed to believe this was a real conversation about selling food stamps for cash and that it was a crime.

In the comments, many people reacted to the video as if it were real. Despite the subtle red flags, hundreds of people vilified women as criminals, some with overtly racist comments, while others attacked government aid programs at a time when national debate was heating up over President Donald Trump's planned cuts to aid programs.

Videos such as the fake interview created with OpenAI's new app Sora show how easily public perception can be manipulated by tools that create alternate realities with a series of simple prompts.

In the two months since Sora's arrival, deceptive videos have proliferated on TikTok, X, YouTube, Facebook and Instagram, say experts who track them. The deluge has heightened vigilance against a new generation of disinformation and fakes.

Most major social media companies have policies that require disclosure of their use of artificial intelligence and broadly prohibit content that is intended to deceive. But these guardrails proved woefully inadequate for the kind of technological leap that OpenAI's tools represent.

While many of the videos are silly memes or cute fake images of babies or pets, others are aimed at stirring up the vitriol that characterizes online political discussions. They are already involved in foreign influence operations, including Russia's ongoing smear campaign against Ukraine. Researchers who have tracked deceptive use say companies have a responsibility to do more to make sure people know what's true and what's false.

“Can they do a better job of moderating content against misinformation and disinformation? Yes, they're clearly not doing that,” said Sam Gregory, executive director of the human rights group Witness, which focuses on technology threats. “Can we do a better job of actively seeking out AI-generated information and labeling it ourselves? The answer is also yes.”

Until now, platforms have relied heavily on creators to disclose that the content they're posting isn't authentic, but creators don't always do so. Platforms like YouTube and TikTok have ways to detect when a video is created using artificial intelligence, but they don't always tell viewers right away.

Companies developing AI tools say they are trying to make it clear to users what content is being generated by computers. Both Sora and Veo, a rival tool from Google, embed visible watermarks in the videos they produce. For example, Sora labels each video “Sora”. Both companies also include invisible computer-readable metadata that identifies the origin of each fake.

TikTok announced it would tighten rules around disclosing its use of AI, apparently in response to concerns about the persuasiveness of fake videos. It also promised new tools that let users decide how much synthetic content they want instead of real content.

YouTube uses Sora's invisible watermark to add a small label to indicate that an AI video has been “altered or synthesized.” “Viewers increasingly want more transparency about whether the content they're watching has been modified or synthesized,” said YouTube spokesperson Jacques Maron.

OpenAI said in a statement that it prohibits deceptive or misleading uses of Sora and will take action against those who violate its policies. The company said its app is just one of dozens of similar tools that can create increasingly lifelike videos, many of which don't have any safeguards or usage restrictions in place.

“AI-generated videos are created and shared across a variety of tools, so combating deceptive content requires an ecosystem-wide effort,” the company said.

A spokesperson for Meta said it is not always possible to label all videos generated by AI, especially as the technology is evolving rapidly.

Aron Yamin, CEO of AI content detection company CopyLeaks, said social media platforms have no economic incentive to limit the spread of videos as long as users keep clicking on them. “In the long term, when 90% of the content traffic in a platform becomes AI, it raises some questions about the quality of the platform and the content,” Yamin said. “So maybe in the long term, there may be more financial incentives to actually manage AI content. But in the short term, that's not the main priority.” – New York Times