TORONTO – At a recent machine learning conference, participants were asked to raise their hands if they were starting to work for AI agents. Many hands shot across the room.
This dynamic showed general sentiment about AI agents throughout the Toronto Machine Learning Association summit. This form of generative AI, which first became widely used last year, is specifically designed for developers and engineers to work together.
You will see the agent's use cases add real value. This is much more than before.
Chris AlekschukAI Makerspace co-founder and CTO
Chris Alexiuk, co-founder and CTO CTO of AI Makerspace, an educational community for building large-scale language modeling (LLM) applications, was the organizer of the conference's AI agent sessions. In an interview with Informa TechTarget, he said that when selecting a session, he wanted to highlight the improvement in ease of use at the technical level of AI agents.
“We'll see agent use cases add real value, which is much more than before,” Alekschuk said.
The meeting held earlier this month highlighted many case studies of agent AI applications, including agents for data management, agent AI governance frameworks, and self-hosted agents. To take advantage of their ease of use, developers build AI agents and turn to practical development for their unique needs.
The technology environment leads applications
AI agents are a form of generative AI that infers and dynamically executes tasks to achieve goals with little or no human intervention.
“They can write code,” said Niels Bantilan, chief machine learning engineer at AI Orchestration Platform Union.ai in an interview. “You can make API calls. You can use a variety of different types of tools.”
Greg Loughnane, co-founder and CEO of AI Makerspace, a supercomputer hub, reflected this in the interview, saying technical tasks such as coding are becoming an important area for AI agents.
Software engineers work in self-contained systems, Loughnane said. Therefore, AI agents in a technical environment can provide higher value (ticket completion, ticket completion, bug fixes) while being less complex than other job features.
“This is like a proof foundation for how to imagine these agent systems in other domains,” added Alexiuk. Seeing the usefulness of agents in software development can help other industries demonstrate the benefits of having an autonomous pharmacist or agent team to accomplish tasks.
In contrast to creating new use cases, the use of agents often arises from existing needs and growth projects.
“Agents grow organically from increasing range,” Alekschuk said. For example, what began as a 2023 Search Charging (RAG) application could be working on today's agent-driven applications.
Many conference sessions highlighted such use cases. In a session on the value of agent AI tools for data needs, Josh Goldstein, solutions architect at open source vector database vendor Weaviate, explained how data task trained AI agents can understand the complex data needs of RAG systems.
“A lot of data is troubling,” Goldstein said. Agent RAG agents can help by acting as data experts to retrieve and aggregate important data.
Weaviate currently has three data agents: Query and Transform Agents are personalization agents used to execute and capture tactical and tactical tasks and user behavior to create real-time recommendations.
These data agents work across domains, Goldstein said. Many operations can be performed when working with vector databases or RAG systems. For example, querying data or creating personalized user responses.
Josh Goldstein, Solutions Architect at Weaviate, highlights how it can help company data agents answer complex data queries at the Toronto Machine Learning Society Summit.
The role of agent AI governance
Like other generator AI technologies, AI agents carry considerable risk. Bantilan cited the use of hallucinations and bad actor threats as their biggest concern.
Other risks inherent in AI agents stem from their differences from other technologies. For example, an AI agent can turn a defined 10-step project into hundreds of steps, depending on how the agent splits the subtasks.
Agents can control many of Union.ai project steps, Bantilan said. “This is a magical place, but it's also a place where many modes of failure can occur.”
For example, he said agents could get stuck in an infinite loop while trying to reason through the problem. There can also be mismatches between tasks and agents, leading to overall allocation of tokens and resource calculations.
AI agents also have direct access to the tools. This adds complexity to the hallucination, Bantilan added. Agents can use the wrong tools for a particular task, leading to incorrect results.
Because of these concerns, Agent AI is using today, and therefore has a low risk with a close loop with humans, Bantilan said. For example, tasks such as coding and research support are more suitable for human validation than complex applications, such as AI agents that interact directly with medical patients.
Organizations implementing AI agents also rely on explicit governance frameworks to mitigate risk. In another session, Ankit Misra, director of AI Governance at Canadian Bank CIBC, highlighted how organizations can reduce risk exposures for AI agents.
AI agents have a unique portfolio of risk, Misra said. This means that agents need a new kind of governance model.
For CIBC, this means governance by design. Misra focused on agent governance best practices, including designing with risk in mind, working with stakeholders, and continuing to monitor workflows.
“Compliance should not be an afterthought,” Mithra said. “It should be guaranteed by the architecture of the system itself.”
Ankit Misra, director of AI Governance at Canadian Imperial Bank of Commerce, describes six key tenets of the six major AI agents of the TMLS Summit CIBC by design.
Considerations when building an AI agent
AI agents may feel new, but they are a natural advancement in technology that has come before, Bantilan said. AI agents are built on familiar features, constructs and primitive building blocks that all programmers can access.
Bantilan said engineers need to remember the basics when building agents: goals, success metrics, raw materials, and data definitions.
“I'm a lot of caring about the quality of the data and the data,” he said. Because LLM is trained with all different types of data, it takes a lot of effort to curate the dataset to train AI agents.
According to Bantilan, when agents are a performance-cost trade-off, another major consideration is the performance-cost trade-off. When agents are produced, costs can easily swell due to scaling and orchestration needs.
Bantilan is recommended to start with an LLM provider that can be called via the API. If the agent is more sophisticated and ready for production, organizations can consider controlling costs, for example, by self-hosting models.
In the session on AI Agent Self-Hosting, Amanda Milberg, a leading solution engineer at Doubleword, a self-hosting AI reasoning platform, self-hosted agent workflows, took participants through the demo using Open WebUI and QWEN3:1.7b.
“Self-hosting an agent system can serve as a strategic advantage,” Milberg said. For one reason, the agent needs a significant amount of tokens. It is difficult to pinpoint how much agent usage costs an organization, especially with recent inference models. In comparison, self-hosting is often a fixed cost.
Secondly, owning an entire AI stack has technical advantages, she said. One such advantage is prefix caching. A typical input-saving technique, without the need for LLM to calculate every time. This is beneficial for cost, delays and performance.
Self-hosting also allows teams to fine-tune their agent models to use cases, Milberg said. And finally, everything stays within the security environment, making it easier for organizations to control compliance.
Amanda Milberg, a leading solution engineer at Doubleword, queries self-hosted agent systems and performs customer checks on TMLS Summit. Agents rely on specific tools they have access to to perform the checks accurately.
Use a build-first approach and remember your domain expertise
Bantilan suggested that developers start by building agents locally from scratch on computers. “We learn a lot about the limitations of these systems where engineering and adhesion of various inputs and outputs must occur.”
That's not to say frameworks like Langchain and Crewai are useless, Bantilan said. They can be a great tool for developers who need fast results. However, all frameworks have limitations.
“They have a happy path and if your requirements turn that happy path, you're your own,” Bantilan said. Therefore, developing agents independently can help developers acquire skills to customize when the framework alone is not enough.
Alexiuk and Loughnane shared similar feelings and recommended a “build-first” approach to learning the fundamentals of agent development. From there, the team can consider custom building AI agents that fit their use case or specialization.
Developers also need to emphasize their own subject matter expertise and data, Loughnane said. The more you understand the details of how a good team contributes to decision-making, the more successful your agents will be.
“Everyone has an agent,” Rahunane said. “What they don't have is your data… your customers [or] Your ability to understand your business operations. ”
Olivia Wisbey is the Associate Site Editor for Searchenterpriseai. Wisbey graduated from Colgate University with a Bachelor of Arts in English Literature and Politics, covering topics in AI, machine learning and software quality.
