The rapid rise of agent AI systems is reshaping expectations around accountability, risk management, and governance across India's financial sector. As autonomous systems take on more complex decision-making roles, the industry is being forced to rethink how to incorporate trust, oversight, and human judgment into AI-driven workflows. These themes played a central role at recent international conferences. AI@Work: Shape the future of business with AI Panel discussion in Mumbai. Nagaraj Nagabhushanam, VP of Data and Analytics and Designated Lead for AI, will moderate the event. hinduism.
The discussion brought together senior technology and risk leaders from across banking, insurance and enterprise technology, including Rohit Kilam, CTO, HDFC Life Insurance. Premraj Avasthi, Head of IT and CIO, GIC Housing Finance Ltd. Pushkal Tenjerla, Head of IT Security, RBL Bank. Rajesh Malhotra, senior leader of data and AI at IBM. Together, they investigated how agent AI is changing not only the operating model but the very foundations of governance in regulated industries.
Rethinking the governance of autonomous agents
The conversation began with an in-depth look at how agentic AI disrupts traditional surveillance structures. Killam sees this change as a fundamental shift in the tempo of governance itself, observing that “we are seeing a shift from slower governance to faster governance.” Autonomous agents are designed to act, learn, and adapt in near real-time, so traditional reactive control is no longer sufficient.
To address this, Kilam outlined three governance models that organizations are currently working on: fully autonomous systems, human-involved workflows, and human-involved audits. Each represents a different balance between machine autonomy and human oversight. However, he stressed that regardless of the model adopted, the principle remains that governance mechanisms must operate at the same speed as the systems they are designed to oversee. Embedded control, continuous monitoring, and real-time intervention are becoming essential features rather than optional safety features.
The enduring relevance of human judgment
While agent AI promises efficiency and scalability, Avasthi emphasized that human judgment is still essential, especially in financial decision-making situations that require nuance and situational awareness. “We need to take reasonable steps to ensure that AI learns aspects of human intent,” he said, highlighting the limits of automation in an area shaped by diverse borrower profiles, regional policies, and socio-economic variables.
In lending and housing finance, decisions are often influenced by unstructured data, behavioral signals, and situational factors that are difficult to fully codify. Avasti argued that even though AI systems assist with analysis and pattern recognition, these complexities still require deliberate human interpretation. Rather than seeing governance as a human-versus-machine binary, he positioned governance as a shared responsibility where oversight is distributed across technology, processes, and people.
Concepts of trust, risk, and compliance
Tenjerla expanded on the discussion by directly linking governance to trust from a risk and security perspective. He cautioned that the success of agent AI in a BFSI environment depends not only on technical robustness but also on operational reliability. “We're not just managing the technology part of agent AI here; we also need to manage the behavioral part of it,” he said, noting the importance of defining acceptable system behavior along with performance metrics.
Tenjerla emphasized that guardrails need to evolve proactively, often ahead of formal regulation, especially as AI systems become more autonomous. He said the quality and freshness of data is non-negotiable and said currency is necessary for responsible decision-making. Outdated or incomplete data can undermine results, erode trust, and amplify risk. In this context, governance becomes a living framework, one that continually adapts to emerging threats, regulatory expectations, and operational realities.
Governance as an embedded foundation
Malhotra expanded the conversation to an architectural level, emphasizing that governance cannot be retrofitted to agent AI systems after deployment. “There's no question that governance needs to be part of the whole process,” he said, advocating for design principles that prioritize accountability from the beginning. He says granular metrics, comprehensive event capture, and end-to-end traceability form the backbone of responsible AI systems.
These capabilities allow organizations to understand not only what decisions an AI system makes, but also why and how those decisions were reached. This level of transparency is important for auditability, regulatory compliance, and long-term scalability, especially as agent systems become more deeply integrated into core business functions.
A clear path forward
The panel concluded with a shared belief that agent AI is not a passing trend, but a tectonic shift in the way intelligence is operationalized across the financial sector. However, its implementation must be guided by discipline, foresight, and humility. As governance evolves from an output of oversight to an intent of oversight, organizations face the dual challenge of innovating quickly while maintaining trust, accountability, and control.
The message from Mumbai was clear. The future of agent AI will not only be determined by the sophistication of the technology, but also by the strength of the governance infrastructure that supports it. In the era of autonomous behavior, trust must be built as rigorously as the system itself.
issued – December 23, 2025 4:10 AM IST
