- Facebook is awash with AI “miscellaneous” images. They’re unrealistic and sometimes grotesque. They’re pure fodder for engagement.
- There's even a Twitter/X account dedicated to posting the most ridiculous stuff.
- So why isn't Facebook cracking down?
There's Jesus' head on a piece of fried chicken. A woman rides a bike with a basket full of babies and burritos.
It's all “AI slop,” a new term for the sudden deluge of AI-generated garbage content, from e-books to viral photos. Slop is all over the internet, but the purest version of it is on Facebook.
This is a shrimp Jesus-like mishmash: weird, obviously fake, and sometimes vaguely unsettling enough to induce trypophobia. Common themes include an old man holding a birthday cake and saying happy birthday, a baby doing something unbaby-like, a bus full of hundreds of babies and other baggage, a snake eating bikes and other vehicles, a soldier with a prosthetic leg, a woman with no limbs and large breasts, and Jesus. The images are often creepily exploitative.
These posts tend to be posted on Facebook pages with generic names and captions that don't necessarily reflect the content of the photo, such as “Why don't these photos go viral?”, “Beautiful flight attendant,” or “Scarlett Johansson” for an image of Jesus built into the hood of a car.
Why sloppy? The best explanation is that the Facebook pages that post these act as bait for potential scam targets, helping scammers identify gullible people in the comments sections and extract money from them.
Another possibility is that posting these images is a great strategy for building a large page audience, and it could be profitable.
The vague similarities between the pages suggest that the AI junk is being posted by a network of shady actors based outside the U.S. This is spam-like content, if not outright spam.
This miscellany is so hilariously miscellaneous at times that there's even an X-account called “Insane Facebook AI Slop” dedicated to cataloging it.
It's embarrassing for Facebook that users of Twitter/X (X!!! A place full of bad stuff!!!) are laughing at Meta's poor content moderation.
This has been going on for a few months now. 404 Media, which has been covering this issue extensively, explains that the images are part of an AI-powered “zombie internet” where bots talk to each other. A while back we speculated about why Meta hasn't stepped up its crackdown, and since nothing seems to have changed, we wonder again…
Why doesn't Facebook ban this rubbish?
Here are some of my theories.
- These images are not nude or otherwise prohibited images and do not technically violate any rules.
- Facebook has created warning labels to let people know if their systems detect images that may have been created with AI.
- Users actually enjoy this content, and the comments section is full of people who have probably been fooled into thinking it's a real image, or that it's simply an interesting artistic expression – why ruin content that people love?
- Facebook bans these posts, but only at the account level if the account posting them is clearly spammy, but not all accounts are spammy.
- Vulgar posts often border on scams, but the accounts posting them aren't necessarily directly committing scams, so they won't be banned.
- Facebook is concerned that banning this kind of AI content would set a bad precedent, as it wants to encourage people to use its Llama 3 tool to create content.
- Facebook isn't a fan of shoddy AI, but it's concerned about a bigger problem.
- Facebook has stopped trying, 404 Media's Jason Koebler speculates. Meta's previous commitments to working with researchers and academics have evaporated, and oversight of content moderation has changed (with Elon Musk lowering the bar to rock bottom). The theory is that Meta is saving money, focusing on the latest shiny thing (AI tools, Instagram, Metaverse), and caring less about Facebook. In other words, they just don't care about Facebook anymore.
I imagine a combination of all of this may be true to some degree. Meta representatives did not respond to questions about why they don't simply ban AI miscellany. (I recently asked Meta's VP of Product why they don't simply ban the miscellany, too, and he changed the topic, apparently not quite sure what I was talking about.)
Facebook has seen a wide variety of spam and scam engagement bait, from minion memes to magician videos to actually dangerous fake news and extremist groups. If the AI miscellany isn't suggesting that people get together in the real world to overthrow the government, Facebook has seen much worse.
