In the new world of artificial intelligence (AI)-driven automation, businesses face a thorny problem: AI systems that confidently generate plausible but inaccurate information, a phenomenon known as “hallucinations.”
As businesses increasingly use AI to make decisions, the risks posed by such fabricated output are becoming clearer. At the heart of the problem are large language models (LLMs), the AI systems that underpin many of the latest tech tools businesses are deploying.
“LLM is built to predict the most likely next word,” Kelwin Fernandes, CEO of NILG.AI, an AI solutions company, told PYMNTS. “Instead of basing its answer on factual reasoning or understanding, it bases its answer on probability theory about the most likely sequence of words.”
Confident but wrong
This reliance on probabilities means that if the training data used to build the AI is flawed, or if the system misinterprets the intent of a query, it can produce responses that are delivered with confidence but fundamentally inaccurate – hallucinations.
“While the technology is rapidly evolving, it's still possible for results to be completely wrong,” PolyAI CTO Tsung-Hsien Wen told PYMNTS.
For businesses, the consequences of acting on hallucinated information are severe. Inaccurate output can lead to incorrect decisions, financial losses, and damage to a company's reputation. There are also thorny questions about accountability when AI systems are involved. “If you remove humans from the process, or if humans hold the AI responsible, who is accountable or liable for mistakes?” asked Fernandez.
Mitigation strategies
As the AI industry grapples with the challenge of hallucinations, various strategies are emerging to mitigate the risk. One such is search augmentation generation (RAG), which feeds AI with curated factual information to reduce the chance of inaccurate outputs. “Having a framework of fine-tuning and safeguards in place can help significantly limit the likelihood of hallucinatory responses,” Wen said.
Roman Eloshvili, founder and CEO of XData Group, highlighted the role of Explainable AI (xAI) technology in reducing the risk of hallucinations. Eloshvili highlighted that methods such as Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive exPlanations (SHAP) can help identify and correct hallucinations early.
“LIME brings clarity to the prediction system and helps to better understand how the AI model arrives at its output, allowing users to identify potential hallucinations,” he told PYMNTS. “SHAP, on the other hand, explains the impact each feature has on the model's prediction and assigns importance to the different features the model uses to make its prediction. By analyzing the importance of these features, users can identify features that may contribute to hallucinations and adjust their models accordingly.”
Other techniques such as self-criticism and thought chaining, which force the AI to rethink and explain its answers, can also help reduce inaccuracies.
Despite these technological advances, Eroshvili stressed that human oversight will remain necessary, especially in critical areas like anti-money laundering (AML) compliance for banks. “Today, in many cases, for example in AML compliance for banks, both human moderation and the explainability of AI tools are considered mandatory,” he said.
Experts warn that it may be impossible to eliminate hallucinations without sacrificing the creative power that makes LLM so powerful.
“Humans aren't the only ones who make mistakes,” Wen says. “Just as we can't guarantee the effectiveness of the human brain 100%, even the most sophisticated language models, running on near-perfect neural network designs, make mistakes from time to time.”
Overcoming the challenges posed by AI illusions will require businesses to take a multi-pronged approach. Maintaining human oversight, implementing fallback processes, and being transparent about AI usage and limitations are all important. As the technology evolves, finding the right balance between leveraging the power of AI and mitigating risks will be an ongoing challenge for businesses across all industries.
