montreal: A video showing people with Down syndrome and urging users to buy from struggling businesses has garnered millions of views, many of them fake and AI-generated.
The crush of posts promoting resin lamps, crocheted handbags, and clay bowls fits into a broader pattern of AI-generated clips seeking to generate sales by evoking empathy for people with disabilities or those considered disadvantaged, such as black people and the elderly.
Advocates and researchers say the practice perpetuates negative stereotypes and harms real people’s lives and businesses.
All over TikTok, YouTube, and Instagram, AI characters are claiming to be bullied for selling crafts in the context of derogatory insults about people with disabilities.
Nathan Rowe, program director for the International Down Syndrome Society, told AFP that the video perpetuates the stereotype that people with genetic conditions caused by extra chromosomes should be pitied.
“They’re preying on people who are probably a little more sympathetic and have a slightly more benign view of Down syndrome,” he says.
These accounts link to suspicious online stores, some of which promote multiple “5-star reviews” that all contain the same filler text.
AFP found that some of the ceramic products first appeared in the content of their actual creators before being appropriated. One of the crochet bag designs may not be handmade and is sold on Shein, a site popular with dropshippers. Shein is an online retailer that operates a storefront without a physical inventory.
Lowe said the videos could “crowd out” the posts of actual Down syndrome entrepreneurs and siphon off business.
“There are a lot of really talented people out there who are making things with Down syndrome, but it kind of reinforces the idea that people with Down syndrome can’t do that, so it has to be AI,” he says.
continuing trend
This is not the first time this trend of exploiting underprivileged people for online attention and profit has occurred.
Down Syndrome International complained to Meta about sexual deepfakes of people with Down syndrome, resulting in many videos being removed. But Lowe said social media companies should be more proactive in preventing the spread of this type of content.
AFP contacted Meta about trends in advertising new products, but did not receive a response. According to TikTok’s community guidelines, the platform prohibits accounts that engage in deceptive or manipulative activities or discrimination. YouTube has a similar policy regarding misleading spam.
Many of the videos examined by AFP are no longer public, but other accounts are still sharing AI-generated clips and redirecting users to the product.
“The people behind the scenes are probably motivated by profit and don’t consider the harm they do in the process,” Lowe said.
Easy to generate
Jeremy Carrasco, co-founder of AI research firm Riddance, said the number of accounts pushing this type of content shows the videos are functioning as a commercial scheme.
“There are a lot of failures across the system that are compounding this,” he said, noting that while videos are very easy to create, they are difficult to track.
“That’s why they’ve exploded so much.”
He said there were countless AI-generated videos featuring people with Down syndrome, and that the same accounts were trying to sell the same products using synthetic elderly characters.
AFP previously investigated the trend of multilingual videos stealing the identities of elderly people in order to lure users into buying slippers or dog collars sympathetically.
“I feel like we’re reaching the bottom of the tolerance range. If they keep going further, something is going to happen,” Carrasco said. –AFP
