I’ve been doing this at work recently, and I feel like people are sometimes surprised when I hear things like “building a CatBoost classifier” or “fitting an isolation forest.” But I actually spend a lot of time on this. My company’s product is an advanced agent AI platform, but we don’t do a lot of prompt engineering on a daily basis.
If you had asked me a year or two ago, I would have been seriously worried about the future of classical ML jobs in the machine learning field. Because we were deep into the paradigm of using the LLM to handle things, regardless of whether the LLM did a good job or not. But it turns out that agent AI probably requires a lot more classical ML than we thought.
In case the reader is unfamiliar, I’ll give a quick summary.
- An AI agent means combining LLM with other software tools to create workflows and coordinate any number of models and tools with minimal or no human intervention.
- LLMs are typically interfaces between human users and all other software tools, and include tasks such as translating human prompts into computer language. LLM also interprets the tool output and chooses which tool to invoke at the appropriate time.
- This enables much more functionality than LLM alone. This is because, as we’ve explained many times here, LLM is just a token generation model, predicting the next word or phrase in a passage based on context.
- Apart from autonomous agents, LLM interfaces need to be combined with other tools to do a huge number of things that we colloquially think of LLM chatbots as doing. ChatGPT, Gemini, and Claude do this kind of thing by coupling LLM interfaces with data searches, web searches, math calculators, and more.
As you can see, a key aspect of the entire agent AI ecosystem is the tools. Agents must have access to tools to effectively complete tasks outside of LLM’s core functionality.
These tools can take all kinds of forms. Many of the tools used in business environments today include data search and organization tools, graph databases, RAG knowledge bases, and query construction and validation.
Why use traditional ML?
However, we would like to remind you that traditional ML models can also be very valuable tools for agents. Rather than just invoking rudimentary tools, we go a step further and enable the use of AI agent models. For example, consider an agent designed for real estate analysis. If you want to know the fair market value of your property, give your agent your address. You can use one API retrieval tool to retrieve details about a property and pass those formatted details to a regression model that generates a price estimate.
Of course, you could theoretically just ask the LLM to estimate the value itself. However, this is questionable and even dangerous for many reasons.
- accuracy: LLM is not particularly suited for tasks that require calculating meaningful numbers. LLM is a guess and does not base its calculations on empirical evidence. A well-trained classical ML model will be much more accurate and reliable.
- interpretability: LLM Interpretability and explainability through speculation is minimal. We know that LLMs tend to be black boxes, which severely limits our ability to assess the path to arriving at the quote we receive. Traditional ML models allow you to identify the decisions made to arrive at an inference and validate it against subject matter expertise.
- Fee: Running an LLM quickly becomes expensive (see my articles from the last few months about token costs). If you have a large number of cases to run, the token price will quickly become meaningful. Running a classifier or regression model is incredibly lightweight and cheap, even at high volumes. Additionally, LLM does not have control over the cost of each call, which can increase token usage and spending. This is already happening across the technology industry.
- accuracy: You have no control over training or tuning the LLM (except to fine-tune the underlying model). You might trust a generic LLM to do the job, but you’re taking a big risk, and as I said above, validating the job is very difficult. On the other hand, fine-tuning the underlying model can be effective, but it requires much more data and more specialized skills than just training a regression or classifier, and interpretability issues remain.
- Data management: Data can leave the controlled environment and be accessed by third-party LLM model providers, creating a potential risk.
- Infrastructure control: With LLM, you no longer have authority over infrastructure management, so third-party downtime poses a risk to your business.
Of course, building a classical model requires different skills than simply setting an LLM to the task. You must understand your data well and be prepared to use your subject matter expertise to complete feature engineering, and you must have sufficient compute and data to train your model. Without labeled data, you are limited to unsupervised learning or bootstrapping your own labels. Fortunately, there is a wealth of content available on how to build these models, how to rigorously evaluate them, and monitor them after deployment.
Connect the model to the agent
This might make sense if you try it, but before you start, you should also consider some architectural choices. How do models and agents interact?
direct phone
Probably the easiest way to start running is to have the model as a tool to call directly into the agent. This is an example format for my real estate research tool. Agents can hit models for just-in-time inference based on prompts. To set this up, you need to enable the AI agent to correctly format requests for legacy models. The agent needs to understand what this model is for, when to call it, and when to use something else. This means clearly documenting the purpose and functionality of your model, which is a familiar task if you’re already building agent AI.
The output must structure the model response so that the AI agent can effectively process it. Simply returning numeric results may not work because the agent needs context information to interpret and make the most of the numeric results. In my models, I often use f-strings as part of the inference to build textual explanations, telling me, for example, what the most important features of the model are and what the probability of the outcome is. Simply returning probabilities limits the agent’s ability to interpret the output and generate useful responses for the end user.
database access
Another option is to make the model a provider of context data rather than a direct tool for the agent. You can precompute inferences by running traditional ML models as scheduled jobs and store these inferences in a data storage solution that agents can access. Rather than making the initial inference call directly to the model API, the agent creates a query and passes it to the database.
This can be a good solution if you have finite cases where you may need to complete the inference. For example, if you have 500 individuals in your database and your agent aims to capture their financial health information, you can use a credit scoring model to precompute their creditworthiness so that it can be captured at runtime along with other data that the agent collects. Depending on your model infrastructure, effective caching of results may reduce latency and repetition.
This approach creates different invocation and retrieval requirements than invoking the tool directly. If you have precomputed inferences and made them available to the agent via a database, the agent needs to be aware that these results exist. If you’re not aware that tables and content are available, you won’t use them if necessary. If the agent calls the database for other information, prompt engineering may already have the infrastructure to tell the agent what the database contains, so reusing it can avoid duplicating effort. If not, the agent may require specific tools that can be used to review database metadata.
Regarding the format of the results, the requirements are similar to those for direct access to the tool. Text-based explanations of results are a good choice because agents need to be able to interpret what they get, regardless of the source.
conclusion
Traditional ML models were state-of-the-art in a variety of industries for more than a decade before the advent of LLM, providing people with insights into their data that they couldn’t get any other way. This power should not be discarded, but instead can be combined with the capacity of the LLM. You can leverage LLM’s strengths in translating human language into computer language, stringing together calls to different tools, and retrieving results from those tools, and use traditional models within this framework to do tasks for which LLM is not well-suited.
The barrier to entry is the skillset to create high-quality classic ML models, which unfortunately isn’t as attractive as modern AI-related jobs. However, the benefits include accuracy, accuracy, interpretability, cost, and control, making it worth the effort. We encourage practitioners to hone their skills using tools such as XGBoost, LightGBM, and scikit-learn to see these results for themselves.
For more information about my work, please visit www.stephaniekirmer.com.
Read more
https://arxiv.org/pdf/2602.14295
https://github.com/Tejas-TA/predikit
https://arxiv.org/pdf/2506.20430
Introduction to Python packages – xgboost 3.3.0 documentation
This document provides a basic tutorial for the xgboost package in Python. The Python package consists of three…xgboost.readthedocs.io
Python Package Overview – LightGBM 4.6.0.99 documentation
This document provides basic instructions for the LightGBM Python package. List of other useful links We recommend…lightgbm.readthedocs.io
Tutorial |
CatBoost includes a wealth of educational material for both beginners and advanced machine learners and data scientists…catboost.ai
Start
Scikit-learn is an open source machine learning library that supports supervised and unsupervised learning. That too…scikit-learn.org
Learn intermediate machine learning tutorials
Handle missing values, non-numeric values, data leaks, etc.www.kaggle.com
todatascience.com/tool-calling-explained-how-ai-agents-decide-what-to-do-next/
todatascience.com/ai-agents-explained-what-is-a-react-loop-and-how-does-it-work/
