The video that OpenAI released to announce their new text-to-video conversion tool Sora has to be seen to be believed. A photorealistic scene of a woolly mammoth charging through a snow cloud, a couple walking through falling cherry blossoms, and aerial footage of the California Gold Rush.
In response to the demonstrations, film producer Tyler Perry reportedly suspended an $800 million studio investment. The logic is that tools like Sora, which promise to transform a user's vision into realistic video with simple text prompts, will make studios obsolete.
Some worry that such artificial intelligence (AI) could be misused by people with dark imaginations. Malicious actors can use these services to create highly realistic deepfakes to confuse and mislead voters during elections, or simply to spread divisive rumors and cause chaos. There is a possibility that
Regulators, law enforcement and social media platforms are already using AI-generated audio, including fake voices of political leaders that allegedly helped distort elections in Slovakia and deter people from voting in the New Hampshire primary. It is struggling to respond to the rise of disinformation.
Politicians and civil society are concerned that as these tools become increasingly sophisticated, it will become ever more difficult for the public to distinguish between what is real and what is fake. .
But political disinformation and AI experts say the increased use of AI products is just a new facet of an old problem. These tools simply add to the already rich array of technologies and techniques used to manipulate and mislead.
Addressing the challenge of deepfakes actually meant addressing the open question of how to regulate the social media platforms where deepfakes spread, leaving their products vulnerable to abuse. This means holding Big Tech companies accountable in some cases.
Callum Hood, head of research at the Center for Countering Digital Hate (CCDH), a campaign group, said: “These AI image generators could exacerbate the problem of election disinformation, which is already a problem.'' We should be very aware of this.” “Existing issues already require stricter measures by social media platforms.”
Several companies that offer generated AI image makers, such as Midjourney, OpenAI, and Microsoft, have policies that prevent users from generating misleading images. However, CCDH claims these policies are not being enforced.
In a study published March 6, the center found that fake photorealistic images of President Joe Biden in the hospital and greeting immigrants could be dangerous in the highly partisan context of the U.S. election. We have shown that it is still relatively easy to generate some possible images. CCTV style footage of Mexican border and election fraud.
These images reflect common falsehoods in American politics. Former President Donald Trump routinely promoted the idea that the 2020 election results were manipulated, a lie that helped spark violent protests at the U.S. Capitol in January 2021. Ta.
“It shows [the companies] We didn’t think about this enough,” Hood said. “The big vulnerability here is in the images that can be used to support stolen election narratives and false claims of election fraud.”
The researchers found significant differences in how individual imagers responded to the prompts. Some image creators did not allow users to create overtly partisan images. “These differences show that it is possible to implement effective safety measures,” Hood said, adding that this reflects a choice on the part of companies.
“This is symptomatic of a broader imbalance between AI companies' profit motive and safety,” he said. “They have every incentive to move as quickly as possible with the least amount of guardrails in place so that they can get their products out there, get new features out there, and get a little bit more venture capital and investment. They have no incentive to slow down and be safe in the first place.”
OpenAI, Microsoft, and MidJourney did not respond to requests for comment.
little accomplished
That incentive is likely to come only in the form of regulation that forces technology companies to act and penalizes them for failure to act. But social media disinformation experts say there's a sense of déjà vu. The conversation we're having around regulating AI is eerily similar to the conversation we had years ago around the spread of misinformation on social media. Big tech companies have pledged to take steps to combat the spread of dangerous falsehoods, but the problem persists.
“It's like Groundhog Day,” said William Dance, a senior research fellow at Lancaster University. He advises UK government departments and security agencies on disinformation. “And it really speaks to how little we have accomplished over the last 10 to 15 years.”
With potentially competitive elections in the European Union, the UK, India, and the US this year, Big Tech companies are taking individual steps to reduce the spread of this type of disinformation and misinformation on their platforms. or renewed their pledge as a group.
In late February, Facebook and Instagram owner Meta announced a series of measures aimed at reducing disinformation and limiting the scope of targeted influence operations during the European elections. These include allowing fact-checking partners (independent organizations that Meta authorizes to label content on your behalf) to label content that is generated or manipulated by AI.
Meta is one of about 20 companies that have signed on to the Tech Accord, pledging to develop tools that can uncover, label, and potentially debunk AI-generated misinformation.
“It seems like there's a kind of blank template that says, 'We're going to do our best to protect you from the blank,'” Dance says. “Disinformation, hate speech, AI, everything.”
The convergence of unregulated AI and unregulated social media is especially likely to require some of the largest social media platforms to take responsibility for overseeing their response to disinformation, misinformation, hate speech, and other harmful content. Many in civil society are concerned because the government is reducing the “trust and safety” team responsible. . X (formerly Twitter) laid off nearly a third of its trust and safety staff after Elon Musk took over the platform in 2022.
“We are in a doubly alarming situation: Spreaders of election misinformation have new tools and capabilities available to them, while social media companies are often “It seems like they are intentionally limiting their ability to act on the information,” CCDH's Hood said. “So this is a very important and worrying trend.”
It will be difficult to predict how prevalent deepfakes will be on social media before the election cycle begins in earnest. But some of the damage has already been done, researchers say. As people become more aware of the ability to create sophisticatedly faked footage and images, a broader sense of distrust and anxiety is being created. Authentic images and videos may be dismissed as fake. This is a phenomenon known as the “liar's dividend.”
“When you go online and you see something, you think, 'Is this real?' Is this generated by an AI? I can't say any more,” says North, who researches AI and disinformation. said Kaichen Yang, a researcher at Eastern University. “For the average user, we think it's going to get even worse. It doesn't matter how much authentic content is online. As long as people believe there's a lot of it, we have a problem.” .”
