RAGE Tools reveals ChatGPT's source

A team of researchers based at the University of Waterloo has developed a new tool, nicknamed “RAGE,” that reveals where large-scale language models (LLMs) like ChatGPT are getting their information from and whether that information can be trusted.

LLMs like ChatGPT rely on “unsupervised deep learning” to make connections and absorb information from across the internet in ways that are difficult for programmers or users to decipher. Additionally, LLMs are prone to “hallucinations” – writing persuasively about concepts or sources that are false or non-existent.

“You can't necessarily trust an LLM to explain itself,” says Joel Rothes, a computer science doctoral student at the University of Waterloo and lead author of the study. “The explanations and citations that they provide may also be fabricated.”

Rothes’ team’s new tool employs a recently developed strategy called “Search Augmentation Generation (RAG)” to understand the context of LLMs’ responses to a given prompt.

“RAG allows users to provide their own sources to LLM for context, and our tool shows how different sources lead to different answers when using RAG, helping users assess whether the information is trustworthy,” Rorseth said.

Their tool focuses on search-enhanced generative explainability, which is why they named it “Wrath at the Machine.”

Understanding where LLMs like ChatGPT are getting their information from and making sure they're not repeating false information will become increasingly important as these tools are adopted in highly sensitive, people-centric industries such as healthcare and the legal sector, Rorseth said.

“Right now, technological innovation is outpacing regulation,” he said. “People are using these technologies without understanding the potential risks, so we need to make sure these products are safe, trustworthy and reliable.”

The research, “RAGE Against the Machine: Search-Enhanced LLM Explanation,” will be published in the Proceedings of the 40th IEEE International Conference on Data Engineering. arXiv Preprint server.