Many expect to use AI agents, but few have governance in place

Organizations with mature agent AI governance frameworks are clearly in the minority, but that hasn’t stopped leaders from rushing to implement the technology in a much broader range of applications this year.

Processing details

This is according to data from four major companies, Deloitte. The current state of AI in the enterprise According to the report, 74% of companies say they plan to implement agent AI within two years, 23% are already using agent AI at least moderately, and 85% of companies expect to customize agents to fit the unique needs of their business. Trust and investment in AI is also on the rise in general, with 84% of organizations increasing their investment in AI and 78% of leaders reporting increased trust in the technology.

At the same time, only 21% of companies say they currently have a mature model for the governance of autonomous agents. This is despite the fact that leaders expect to deploy the technology in a wide range of areas, including real physical systems such as package sorting, manufacturing, and security systems. Physical AI integration is already growing, with 58% of companies reporting at least limited use of physical AI. This proportion is expected to reach 80% within two years. Deloitte noted that physical systems come with inherent risks that need to be considered.

“Unlike software AI that operates in digital environments, physical AI systems interact with people, equipment, and infrastructure in ways that can pose safety risks if they malfunction or are compromised. This increases the risk of testing, certification, and continuous monitoring,” the report states.

Governance issues may also challenge another growing concern among leaders: where AI is coming from. Deloitte’s report notes that the specific countries in which its AI solutions have been developed are becoming increasingly prominent. This relates to the concept of “sovereign AI,” which is when countries and companies operating within them design, train, and deploy AI on infrastructure they control, using locally controlled data, based on their own laws. The goal is to reduce dependence on foreign vendors for critical AI capabilities.

According to the report, 83% of companies consider sovereign AI to be at least moderately important to their strategic plans, and nearly half (43%) rate sovereign AI as very or very important. Meanwhile, 66% are at least somewhat concerned about over-reliance on foreign AI, and 22% are very or very concerned.

“The rise of Sovereign AI has immediate practical implications. Companies that operate across borders will have to deal with complex requirements that differ from country to country, and will need to build solutions tailored to different markets.…At the end of the day, Sovereign AI “It doesn’t just mean technology ownership; it means strategic independence. By building on an infrastructure within their control (with their own data, models, people, and ecosystems), companies gain the ability to innovate securely and responsibly.”

Deloitte’s report is similar to several others that, taken together, show that there remains a gap between the use of AI and the governance that oversees it. For example, in July, AuditBoard poll found More than 80% say their organization is very or very concerned about AI risks, but only 25% say they have fully implemented an AI governance program.

another August survey by Eisner Amper Only 22% of respondents said their organization monitors the use of AI in the first place, and only 11% block ChatGPT and other public models. The survey also found that only 36.2% of companies have an AI policy, 34.2% say their company values ​​transparency when discussing AI, and only 34% say their company has an AI strategy.

We may see some progress in this regard this year. Many accounting technology leaders predicted AI governance will find new relevance and strength in 2026.