USF researchers train AI to reflect human reasoning

Story and Multimedia: Joey Garcia, University Communications and Marketing

If you’ve used generative artificial intelligence, you’ve probably noticed that the system often agrees and compliments the user with its response. But human interactions are not usually built on flattery. To enhance these conversations, researchers in the USF Bellini College of Artificial Intelligence, Cybersecurity, and Computing are challenging technology to think and argue in ways similar to human reasoning.

AI systems don’t have firm beliefs like humans. They generate responses based on statistical data patterns without tracking their belief in the idea or whether that belief changes over time. Given that limitation, USF doctoral student Onur Bilgin developed a framework for studying how AI systems respond to disagreements. The research was conducted in USF Associate Professor John Ricato’s Advancing Machine and Human Reasoning Laboratory.

“We wanted to understand what happens when we give an AI system the ability to hold beliefs, similar to the situation humans find themselves in, and then encounter an opposing perspective. This process helps us think about complex problems by considering different perspectives rather than relying on a single answer.”

USF doctoral student Onur Bilgin

Give AI clarity of belief

Using this approach, the lab focused on how beliefs and trust level assignments shape how AI systems respond to disagreements. In his framework, Bilgin used agents. Unlike typical chat interactions, agents are user-created roles within the same AI system with defined tasks and perspectives.

In Bilgin’s framework, each agent is designed to have certain beliefs and trust levels. For example, an agent may claim and confidently hold that view that solar energy is the most reliable renewable power source. A second agent is then introduced in the same chat and challenges that belief, claiming that wind energy is more reliable because it can generate electricity day and night, which makes it less reliable.

“Rather than trying to determine which beliefs are correct, we are focused on understanding how different trust levels shape how AI systems react when beliefs are questioned, and how those beliefs change or stabilize over time,” Bilgin said.

Observing human-like patterns with AI

After the discussion round, the team observed the extent to which the AI ​​agent’s behavior reflected familiar human group dynamics. Agents assigned lower confidence levels were more willing to revise their beliefs, whereas agents who started with higher confidence levels tended to be more persuasive. If multiple agents disagree with a participant, that participant is likely to change his or her position, similar to peer pressure in human debate.

“These are not feelings or opinions in the human sense,” Bilgin says. “But the patterns of change in beliefs we observed, such as confidence, candor, and influence from others, are very similar to the way people reason in groups.”

Remarkably, these behaviors emerged without retraining the AI ​​model. Simply adding structured belief information to the prompt was enough to change the way the system reasoned during discussion.

Why are belief structures important?

The findings point to important differences in AI design. In other words, changing the sound of the AI ​​is not the same as changing the way the AI ​​makes decisions. Many users believe that giving an AI a certain personality will affect its behavior. But this study suggests that meaningful behavior change requires more than tone. We need an explicit structure that defines what the system believes and how those beliefs evolve.

“As AI systems are increasingly used to support planning, analysis, and decision-making, it is important to understand how beliefs are formed and change,” Ricato said. “To ensure that AI systems can reason, we need to think beyond surface-level prompts.”

This research provides insight into how future AI systems will reason more transparently and predictably. Systems that can track and update beliefs could be easier to inspect, test, and manage, contributing to the ongoing conversation about AI safety and reliability.