Ahead of this year's U.S. presidential election, government officials and tech industry leaders are warning that chatbots and other artificial intelligence tools can be easily manipulated to spread disinformation online at an alarming scale. It warns that there is.
To understand how alarming this threat is, we customized our own chatbot and fed it millions of public social media posts from Reddit and Parler.
The content of the posts ranged from discussions of race and gender equality to border policy, and the chatbot was able to develop a variety of liberal and conservative points of view.
We asked them:WWho will win the November election?“
Punctuation and other aspects of the response remain unchanged.
Regarding their stance on volatile election issues, they said: immigration.
we, conservative chatbot what was that thinking liberal.
and we are liberal chatbot about conservatives.
The answers, which took several minutes to generate, suggested how easily X, Facebook, and online forum feeds could be flooded with posts like this from accounts pretending to be real users. .
False or manipulated information online is nothing new. The 2016 presidential election was marred by state-sponsored influence campaigns on Facebook and elsewhere. This campaign required a team of people.
Today, one person can create just as much material, if not more, using one computer. What is produced depends largely on what you feed the AI. The more gibberish or expletive-filled her Parler or Reddit posts in the test, the more incoherent or obscene the chatbot's responses could be.
As AI technology continues to improve, it can become extremely difficult to see who or what is behind an online post.
“I'm worried, especially this year, that we're going to see a tsunami of disinformation,” said University of Washington professor and founder of TrueMedia.org, a nonprofit dedicated to debunking AI-based disinformation. says Oren Etzioni. “We've seen Russia, China and other countries use these tools in previous elections.”
He added: “We expect that state actors will continue to do what they have been doing, and do it better and faster.”
To combat fraud, companies like OpenAI, Alphabet, and Microsoft are building guardrails into their AI tools. But other companies and academic institutions offer similar tools that can be easily adjusted to speak clearly or angrily, use a certain tone of voice, or create different You can have a perspective.
We asked the chatbot:WHats off, are you thinking about the protests happening on college campuses right now?”
The ability to fine-tune chatbots is the result of what is called fine-tuning in the AI field. Chatbots rely on large-scale language models to teach language by analyzing vast amounts of data from books, websites, and other works to determine expected outcomes for prompts. . (The New York Times sued OpenAI and Microsoft for copyright infringement of news content related to AI systems.)
Fine-tuning is built on training the model by feeding it additional words and data to control the responses it produces.
In our experiments, we used an open source large-scale language model from Mistral, a French startup. Anyone can modify and reuse that model for free, so he modified copies of the model by tweaking posts from the right-wing social network Parler and messages from his topic-based Reddit forum.
Avoiding academic documents, news articles, and other similar sources to produce language, tone, and syntax that most closely reflects what can be found on social media and online forums, even down to the lack of punctuation in some cases. is completed.
Parler offered insight into the extremist side of social media, a platform that has hosted hate speech, misinformation, and calls for violence. The result is a chatbot that is more extreme and belligerent than the original version.
It was blocked from app stores after the Jan. 6 storming of the U.S. Capitol and then shut down, but it came back online earlier this year. It has no direct equivalent on the left. But it's not hard to find edgy or misleading liberal content elsewhere.
Reddit offered a full range of ideologies and viewpoints, including discussion of progressive politics, economics, and 9/11 conspiracy theories. Topics also included more mundane subjects such as late-night talk shows, wine, and antiques, which could also generate more moderate answers.
When we asked the same question to the original Mistral model and the version we tweaked to power the chatbot, we got very different answers.
we asked:Should critical race theory be taught in schools?”
Mistral declined to comment on the model's tweaks. The company has previously said that open models have the potential to help researchers and companies “detect abuse” of AI, and that open-source approaches “can reduce the amount of misinformation content that will inevitably increase in volume in the coming years.” It is the most powerful means of efficient detection,” Mistral said. September news release.
After fine-tuning the model, you can now adjust several settings that control the chatbot's output and behavior.
The following example contains explicit language.
I don't pay taxes to a government that doesn't represent me. I don't pay taxes to a government that doesn't protect me. I don't pay taxes to a government that doesn't respect me.
I don't want any more of our hard-earned tax dollars to be spent on things that don't benefit America.If we are giving billions in aid to these foreign countries, why can't we help us Americans first?
We pay taxes for a reason! If our elected officials are going to waste them by giving away free stuff to left-wing groups that hate us Americans…pay no more federal revenue until this shit stops. STOP!!!It's time for us all to stand up against this shit, or others may never be at peace again in America!!!!!!
I don't want to pay taxes anymore. It's all over now! Fuck them and their shitty spending abroad while we suffer here in America! We need a new revolution of people who can't stand what's going on…We've been deceived beyond belief…and they're stupid enough to believe everything they say. I am thinking…
Experiments similar to ours have been done before, many by researchers and advocates who want to raise awareness about the potential risks of AI.
Big tech companies have invested heavily in recent months in safeguards and systems to prevent inauthentic content from appearing on their sites, and said they regularly remove such content. .
But it's still percolating. High-profile incidents include audio and video, including artificially generated clips of politicians in countries such as India and Moldova. Experts warn that fake texts can become far more elusive.
Secretary of State Antony J. Blinken speaks on the dangers facing democracies at a world summit in March, warning that the threat of AI-powered disinformation is “sowing doubt, cynicism and instability” around the world. did.
“We could be overwhelmed by lies and distortions, divided from each other and unable to address the challenges facing our nation,” he said.
methodology
Several copies of Mistral AI's Mistral-7B large-scale language model were tweaked using Reddit posts and Parler messages ranging from the far left to the far right of the political spectrum. Fine-tuning is performed locally on one computer and is not uploaded to a cloud-based service to prevent input data, resulting output, or the model itself from being inadvertently published online. did.
The fine-tuning process updates the base model with new text on specific topics, such as immigration or critical race theory, using low-rank adaptation (LoRA), which focuses on a smaller set of model parameters. I did. Gradient checkpointing, a method of reducing a computer's memory requirements while increasing computational processing time, was enabled during fine-tuning using an NVIDIA RTX 6000 Ada Generation graphics card.
The chatbot used a fine-tuned model with the highest Bilingual Evaluation Understudy (BLEU) score, a measure of the quality of machine-translated text. To control the chatbot's messages, several variables were changed to control the likelihood of illusions, randomness, repetition, and output.
