Meta found deceptive content, “likely AI-generated,” on Facebook and Instagram, including comments praising Israel's handling of the Gaza conflict, which appeared beneath articles from global news outlets and statements from U.S. lawmakers.
In its quarterly security report, Meta explained that these fraudulent accounts posed as Jewish students, African Americans and other concerned citizens and primarily targeted users in the United States and Canada.
The campaign was linked to STOIC, a Tel Aviv-based political marketing firm that claims to act as a “one-stop shop for campaign management of any scale.” According to the company's LinkedIn page, the company offers “data enrichment, online and offline microtargeting, digital asset management and more.”
This is the first time Meta has acknowledged the use of text-based generative AI in influence operations – it had identified AI-generated profile photos in similar situations since 2019.
Generative AI, which can quickly and cheaply generate human-like text, images and audio, has raised concerns among researchers about its potential to spread disinformation and influence elections. Meta's security executive assured that the Israeli campaign was quickly dismantled and that new AI techniques did not hinder efforts to disrupt such influence networks.
They noted that while generative AI tools enable faster and more mass content creation, they have not impacted Meta's detection and countermeasures.
“There are some examples of these networks using generative AI tools to create content, presumably allowing them to create it more quickly or in greater volume, but it hasn't really impacted our ability to detect it,” said Mike Dvilyanski, head of threat research at Meta.
Quoted from Reuters
