OpenAI’s Sora AI video generator has sparked a disturbingly disappointing trend

what happened: This is pretty scary. More and more people are finding AI-generated videos of women and girls being strangled on TikTok and X (the apps that are still called Twitter).

• These are not comics. They look scary and realistic. The clips are perhaps 10 seconds long, have awful titles like “Teenage girl cheerleader strangled to death,” and show a synthetic “woman” crying and struggling.

• Investigators note that many of these videos include a watermark from OpenAI’s new video generator, Sora 2, which was just released to the public on September 30th. Some of the new uploads don’t have watermarks, which is even more scary. This means that people may be wondering how to remove the watermark or use other AI tools to create this stuff.

• The account posting is also creepy. While X’s one had few followers, a similar one on TikTok had more than 1,000 before it was eventually deleted. Apparently they both first started posting actual TV clips in October, but then… they switched to this AI-generated violence.

Why this is important: This is a major red flag. This shows that these AI platforms and social media sites have no idea what “graphic or violent content” is, even though their own rules specifically prohibit it.

• TikTok deleted the account after the media started asking questions, but X reportedly didn’t delete it even after users reported it.

• It also illustrates the darkest risks of these new AI tools. This means it is becoming frighteningly easy for anyone to create surreal videos of violence, especially against women and children.

• This is not just “breaking the rules.” This is completely unethical and raises major legal questions regarding the use of AI to create and share abuse.

Why you should care: Now that anyone can create something like this, it can spread like wildfire and numb people to how horrific this kind of violence actually is.

• It is important to note that although these videos are “fake,” they can still cause real psychological harm and promote a culture of violence.

• This is yet another major wake-up call that we desperately need stronger safeguards. The companies building these AI tools and the platforms that popularize them must be held accountable.

What’s next: The UK government’s recent announcement that it will outlaw all pornographic depictions of strangulation is a sign that countries are starting to realize this.

• Meanwhile, pressure is mounting on OpenAI, TikTok, and X to work together to put real guardrails on their AI tools so they can’t be used to mass-produce this kind of disturbing and violent junk.

• Experts warn that if we don’t act quickly, this is just the beginning of a new wave of AI-powered harassment online.