# introduction
large language model (LLM) tend to use “flowery” and sometimes overly verbose language in their responses. Asking a simple question can result in a flood of overly detailed, enthusiastic, and complex prose sentences. This normal behavior is rooted in training and optimized to be as helpful and conversational as possible.
unfortunately, redundancy It can be argued that this is a serious aspect that should be kept under the radar and is often correlated with an increased likelihood of serious problems occurring. hallucination. The more words you generate in your response, the more likely you are to move away from grounded knowledge and into the “art of fabrication.”
This means you need robust guardrails to prevent this double-sided problem, starting with redundancy checks. In this article: text statistics A Python library that measures readability, detects overly complex responses, and forces models to improve responses before they reach the end user.
# Setting a complexity budget using Textstat
The Textstat Python library can be used to calculate scores such as the Automatic Readability Index (ARI). Estimate the grade level (learning level) required to understand the text, including model answers. If this complexity metric exceeds a budget or threshold (such as 10.0, which corresponds to a 10th grade reading level), a re-prompting loop is automatically triggered, requesting a more concise and simple response. This strategy not only dispels flowery rhetoric, but may also help reduce the risk of hallucinations, as the model adheres more closely to the core facts as a result.
# Implementing the LangChain pipeline
Let’s see how to implement the above strategy and integrate it. rung chain Pipelines that are easy to run in Google Colab notebooks. You will need it. hug face API tokens can be obtained for free at https://huggingface.co/settings/tokens. Create a new “secret” named . HF_TOKEN In Colab’s left menu,[秘密]Click the icon (looks like a lock). Paste the generated API token into the Value field and you’re good to go.
First, install the required libraries.
!pip install textstat langchain_huggingface langchain_communityThe following code is specific to Google Colab and may need to be adjusted accordingly if you are working in a different environment. We focus on recovering stored API tokens.
from google.colab import userdata
# Obtain Hugging Face API token saved in your Colab session's Secrets
HF_TOKEN = userdata.get('HF_TOKEN')
# Verify token recovery
if not HF_TOKEN:
print("WARNING: The token 'HF_TOKEN' wasn't found. This may cause errors.")
else:
print("Hugging Face Token loaded successfully.")The following code performs several actions. First, we’ll set up a component to generate local text via a pre-trained Hugging Face model. in particular, distilgpt2. The model is then integrated into the LangChain pipeline.
import textstat
from langchain_core.prompts import PromptTemplate
# Importing necessary classes for local Hugging Face pipelines
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from langchain_community.llms import HuggingFacePipeline
# Initializing a free-tier, local-friendly, compatible LLM for text generation
model_id = "distilgpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
# Creating a text-generation pipeline
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
max_new_tokens=100,
device=0 # Use GPU if available, otherwise it will default to CPU
)
# Wrapping the pipeline in HuggingFacePipeline
llm = HuggingFacePipeline(pipeline=pipe)Next, core mechanisms for measuring and managing redundancy are implemented. The following function generates a summary of the passed text (assumed to be an LLM response) and attempts to ensure that the summary does not exceed a complexity threshold level. Note that when using the appropriate prompt template, a generative model similar to the following is used: distilgpt2 It can be used to obtain summaries of text, but the quality of such summaries may not match that of heavier, summarization-focused models. We chose this model for its reliability of local execution in a constrained environment.
def safe_summarize(text_input, complexity_budget=10.0):
print("\n--- Starting Summary Process ---")
print(f"Input text length: {len(text_input)} characters")
print(f"Target complexity budget (ARI score): {complexity_budget}")
# Step 1: Initial Summary Generation
print("Generating initial comprehensive summary...")
base_prompt = PromptTemplate.from_template(
"Provide a comprehensive summary of the following: {text}"
)
chain = base_prompt | llm
summary = chain.invoke({"text": text_input})
print("Initial Summary generated:")
print("-------------------------")
print(summary)
print("-------------------------")
# Step 2: Measure Readability
ari_score = textstat.automated_readability_index(summary)
print(f"Initial ARI Score: {ari_score:.2f}")
# Step 3: Enforce Complexity Budget
if ari_score > complexity_budget:
print("Budget exceeded! Initial summary is too complex.")
print("Triggering simplification guardrail...")
simplification_prompt = PromptTemplate.from_template(
"The following text is too verbose. Rewrite it concisely "
"using simple vocabulary, stripping away flowery language:\n\n{text}"
)
simplify_chain = simplification_prompt | llm
simplified_summary = simplify_chain.invoke({"text": summary})
new_ari = textstat.automated_readability_index(simplified_summary)
print("Simplified Summary generated:")
print("-------------------------")
print(simplified_summary)
print("-------------------------")
print(f"Revised ARI Score: {new_ari:.2f}")
summary = simplified_summary
else:
print("Initial summary is within complexity budget. No simplification needed.")
print("--- Summary Process Finished ---")
return summaryNotice also that the code above calculates the ARI score to estimate the complexity of the text.
The final part of the code example tests the previously defined function, passes sample text and a complexity budget of 10.0, and prints the final result.
# 1. Providing some highly verbose, complex sample text
sample_text = """
The inextricably intertwined permutations of cognitive computational arrays within the
realm of Large Language Models often precipitate a cascade of unnecessarily labyrinthine
lexical structures. This propensity for circumlocution, whilst seemingly indicative of
profound erudition, frequently obfuscates the foundational semantic payload, thereby
rendering the generated discourse significantly less accessible to the quintessential layperson.
"""
# 2. Calling the function
print("Running summarizer pipeline...\n")
final_output = safe_summarize(sample_text, complexity_budget=10.0)
# 3. Printing the final result
print("\n--- Final Guardrailed Summary ---")
print(final_output)Although the resulting output message can be quite long, you will notice that the ARI score decreases slightly after calling the pre-trained model for summarization. However, don’t expect miraculous results. Although the selected model is lightweight, it is not good at text summarization, so the reduction in ARI score is quite modest. You can try using other models like google/flan-t5-small Be careful to check the performance of text summarization. These models are heavier and more difficult to run.
# summary
This article shows how to implement an infrastructure to measure and control overly redundant LLM responses by invoking an auxiliary model to summarize the responses before approving the level of complexity. Hallucinations occur as a highly redundant byproduct in many scenarios. Although the implementation presented here focuses on evaluating redundancy, there are also specific checks that can also be used to measure hallucinations, such as semantic consistency checks, natural language inference (NLI) cross-encoders, and LLM-as-a-judge solutions.
Ivan Palomares Carrascosa I am a leader, writer, speaker, and advisor in AI, machine learning, deep learning, and LLM. He trains and coaches others to leverage AI in the real world.
