How fake NZ news pages are taking over Facebook with AI slop

analysis – Misleading “fake NZ news” pages are filling social media, including the use of AI to animate still photos of Mt Maunganui landslide victims.

Dodgy Facebook pages dedicated to posting tons of AI-generated images and videos are now almost unavoidable on the site, but they still scam a huge number of people.

An investigation I conducted for the Australian Associated Press found that a Facebook page calling itself NZ News Hub (which has no connection to the now-defunct Newshub) publishes dozens of posts a week quoting legitimate reporting from news organizations such as RNZ, the New Zealand Herald and Stuff, and supplementing them with sloppy AI-generated images and videos.

In one case, a still photo of a 15-year-old Mt Maunganui landslide victim was grotesquely animated to appear to be dancing.

The page’s profile says, “NZ News Hub brings you the latest New Zealand news, breaking news, politics, business, sport and community updates,” but it doesn’t appear to include any original reporting.

For example, on Waitangi Day, the page featured a post that appeared to be a video of Prime Minister Christopher Luxon at Waitangi, which was clearly generated by AI.

Despite this, the page, which has nearly 5,000 followers, has dozens of people liking and commenting on the posts as if they were real. Many of their followers appear to be business pages, including several politicians.

AAP contacted NZ News Hub for comment but did not receive a response.

Andrew Rensen, senior lecturer and program director in AI at Victoria University of Wellington, said the images on the Newshub page were clearly generated by AI.

“These pages want to get as much engagement (reactions, comments, shares) as possible to increase followers, exposure, and potential advertising revenue,” he told me in my AAP article.

The easy availability of AI tools has made it possible for just about anyone to set up a “fake news” factory with little coordination from big tech companies to prevent a flood. Many fake images display a “SynthID” watermark indicating that Google’s AI tools were used to create them, but you need to know how to spot that watermark in the first place.

As RNZ previously reported, the Mount Maunganui disaster, which killed six people, sparked a surge of AI online.

A separate AAP investigation has found that many images circulating on social media bear no resemblance to the actual scene.

Misinformation about victims is also widespread.

NZ News Hub posted a still photo of victim Sharon Maccanico, 15, dancing, provided to the media by New Zealand Police, and appears to show the teenager performing an almost impossibly acrobatic dance set to an upbeat soundtrack.

NZ News Hub has taken many recent articles from RNZ and reposted much of the coverage with misleading AI adjustments.

One post featured a recent press conference by Luxon and Finance Minister Nicola Willis, using still images taken by an RNZ photographer to create an AI video of them talking about the upcoming November election.

Another recent post referenced RNZ reporter Tim Brown’s recent article about Tauranga parents mourning their daughter’s suicide, but used AI to animate RNZ’s images to appear as if the couple were smiling at each other. Mr Brown confirmed that no such video was filmed by RNZ.

Many images have AI signatures, but others are difficult to identify. Still, it’s usually communicated.

An image purporting to be of police officers responding to recent anti-immigration protests in Auckland shows all the “police” wearing the wrong uniform and their faces extremely blurred and distorted. Meanwhile, another photo claiming to be from a police operation to rescue drowning victims in the Mahurangi River last month shows so-called police with “POPFIL” written on the back of their uniforms, and the river shown bears no resemblance to the real river.

Even New Zealand’s iconic kereru isn’t immune to AI tinkering. An article published by RNZ last week about the dangers of native brown pigeons flying into windows was featured, but the image was inexplicably replaced by an AI-generated bird that looked nothing like a kereru.

Some of the AI ​​images may be used to avoid copyright disputes by stealing photos from legitimate news websites, Rensen said.

“Trust in journalism is already a huge concern, and any source that identifies itself as a fake news source will only further increase distrust,” Rensen told me.

Many fake Facebook pages also flood articles about celebrities with enticing headlines that promise “the truth is out” and scandalous information that is rarely backed up by actual copy. Such articles often link to external websites filled with pop-up ads in an attempt to increase traffic, clicks, and ultimately earn revenue.

This ‘NZ News Hub’ does not exist alone on a slope. Almost every news event imaginable, whether it’s the shark attack in Australia, the capture of Venezuelan President Nicolas Maduro, or the assassination of Charlie Kirk, has seen fake news posts and images flood social media within minutes of the incident.

According to Facebook Page Transparency information, many of these pages actually operate in countries far from where the news originates, such as Vietnam or Malaysia.

Even if Facebook were to take action against these specific pages, dozens of copycat sites could quickly pop up.

Multiple sites are already sharing fake NZ AI news images. Of course, fake images and videos are also widely seen on TikTok, Instagram, X, and more.

This is part of a general decline in social media platforms that author Cory Doctorow has labeled “encitization.”

Mr Rensen said he felt pages such as the NZ News Hub showed that traditional media should be very cautious about using AI in reporting.

“As AI slants become more and more prevalent online, people may return to these established platforms as trusted sources of information. But where do we turn for the truth when established platforms also use AI?”

RNZ has a set of AI principles available online, and we generally do not publish, broadcast or knowingly distribute works created by generative AI.

Many of the NZ News Hub posts have been reported to Facebook owner Meta, but are still up at the time of writing.

Sign up for Ngā Pitopito Korero. Our daily newsletters, hand-picked by our editors, delivered straight to your inbox every weekday.