Social media company Meta said in its quarterly security report that it had found deceptive use of supposedly AI-generated content on its Facebook and Instagram platforms, including comments about the Israel-Gaza conflict being posted underneath posts from international news organizations and U.S. lawmakers.
The social media accounts, posting as Jewish students, African Americans and other concerned citizens, are targeted at audiences in the United States and Canada. The campaign is being run by Tel Aviv-based political marketing firm STOIC. Reuters report.
While Meta has encountered AI-generated basic profile photos in influence operations since 2019, this report is the first to uncover the use of text-based generative AI technology, which has been around since late 2022.
The researchers warned that generative AI's ability to generate human-like text, images and audio quickly and cheaply could lead to more effective disinformation campaigns and influence elections.
Metasecurity executives reported that they did not believe advanced AI techniques would hinder their ability to quickly dismantle the Israeli campaign and disrupt influence networks, which are coordinated efforts to spread a specific message.
The executives added that they have not seen any of the networks deploy AI-generated images of politicians that are so realistic they could be mistaken for real photographs.
“There are several examples of these networks likely using generative AI tools to create content,” said Mike Dvilyanski, head of threat research at Meta.
“Maybe it allows them to do it faster or in greater quantities, but it doesn't have much of an effect on our ability to detect them,” Mike said. Reuters.
In addition to the STOIC network, Meta also shut down an Iran-based network focused on the Israeli-Hamas conflict, although it did not confirm that generative AI was used in that campaign.
It's worth mentioning that Mehta and other tech giants have been working on ways to address the potential misuse of AI technology, especially in elections.
The researchers found instances where image generators created photos containing voting-related misinformation, despite companies like OpenAI and Microsoft having policies banning such content.
The companies emphasize that they will use digital labeling systems to identify AI-generated content as it is created, but these tools do not apply to text, leaving researchers to question their effectiveness.
(Source: Reuters)
You are on Mint! India's No.1 News Site (Source: Press Gazette). For more of our business coverage and market insights, click here!
