There is significant regulatory uncertainty in global AI surveillance, primarily because of the country's fragmented legal landscape, as it hinders the effective governance of global AI systems. For example, as mentioned in the 2024 Natural Studies, the lack of harmonious international law complicates AI innovation, making it difficult for organizations to understand which standards apply in different jurisdictions.
Without a robust AI governance and risk management framework, organizations are exposed to operational, ethical and financial risks. Failure to compliance is expensive. Fines under the EUAI law could reach up to 40 million euros or 7% of global revenue for serious violations.
In a recent episode of the “AI in Business” podcast, Matthew Demello, editorial director at Emerj, sat down with Michael Berger, Head of Insure AI at Munich Re, discussing how to proactively manage growing AI risks by setting up a governance framework, defining risk tolerance, and reducing total risk through model diversification and task-specific fine-tuning.
This article unveils two key insights that every organization needs for effective AI governance.
- Building governance and accountability for AI risks: Define clear risk ownership and implement a governance framework to manage inevitable AI errors across jurisdictions.
- Manage AI risks with governance and model strategies: Defining risk tolerance, implementing cross-regulation mitigation, and diversifying model architectures to reduce systematic bias and cohesion risk.
guest: Michael Berger, Head of Insurance AI, Munich RE
Expertise: Insurance, technology, data management, and technology-based risk assessment.
Simple recognition: Michael has spent the past 15 years at Munich RE, helping to shape AI insurance business. He holds a Masters in Information and Data Science from Berkeley, California, a Masters in Business Administration and a Doctor of Finance from Vandeswale University Munich.
Building governance and accountability for AI risks
Michael opens up the conversation by comparing how the EU and the US approach AI regulation.
- The EU will create regulations in advance and set clear rules and requirements before any issues arise.
- The United States often shapes its approach through litigation. In the lawsuit, court cases emerged over time with precedents and best practices.
For global companies, the difference means that AI deployments must be adapted to the requirements of each jurisdiction. This increases the burden of compliance, but also encourages clearer thinking about risks.
He cites the Canada example in which passengers asked airline AI-powered chatbots about discount policies. The model hallucinated the false policy, passengers relied on it, and the airline refused to respect it. The court ruled the airline liability despite not building a model.
Michael says such cases will clarify who is responsible for AI output, help businesses improve risk management, decide where and where to adopt AI confidently and in the end the growth of a healthier AI industry.
He argues that the responsibility for AI-related errors, especially hallucinations from generative AI, is not influenced by end-users or AI decisions, and should not be heavily placed on AI employers and potentially AI developers.
He explains that generative AI offers a great advantage for many use cases, but these models are inherently probabilistic and imply that they work with possibilities rather than certainty. Due to the inherent biases of probabilistic models, impairments and hallucinations are not only possible, but inevitable, and technical modifications cannot rule them out.
“As a business leader, I think these models are probabilistic models and we need to accept that we can always fail, as there is always probability.
It's not a hallucination failure, this cannot be avoided by technical means. I think that if we do that, we need to embrace this type of risk and accept the greater risk, along with the potential benefits these models create. ”-Michael Berger, Insure AI Head at Munich Re
Manage AI risks with governance and model strategies
Michael points out that the discussion about AI has matured because it sees AI as a distant possibility for many companies to perceive it as a current operational reality. Shifts have a more accurate understanding of AI possibilities that always involve risks, and that risk must be actively managed.
He says this new understanding has led to increased conversations about AI governance, or the way AI is managed, taking operational risks. These conversations include defining an organization's risk tolerance level, implementing mitigation measures that reduce risk to acceptable levels beyond regulatory requirements, and considering AI insurance as part of a strategy to cover potential liabilities.
He says that at the single-company level, if more AI use cases are developed and more interactive AI models are led to production, the overall risk increases. Each additional model results in the possibility of errors and hallucinations. This can lead to liability and financial costs.
He also points out that risks are more severe in sensitive use cases where private consumers are directly affected by AI decisions. In such a scenario, the issue of AI-driven discrimination becomes important.
“I think that is a big change in risk here because discrimination cases were more rare or at least not systematic because people were making decisions before.
However, as AI models are fully used and are affecting many people today, the risk of discrimination is the risk of suddenly becoming systematic. Therefore, if AI models are found to be discriminatory, they could affect many consumer groups in which they are used – not just in a single company, but potentially across the company. ”-Michael Berger, Insure AI Head at Munich Re
Human decision-making may include discrimination, but it is often not systematic. However, in AI, if a model is biased, the bias can be consistently applied at large scale and create systematic discrimination that affects large groups.
Michael further explains that risk can extend beyond a single company, especially when the underlying model is involved. Discriminatory effects can be widely popular if the underlying model embeds discriminatory patterns and many companies are trained in ways that can be used for similarly sensitive applications.
Embedding discrimination into the basic model creates what he calls “aggregation risk.” In this case, defects in one model can cause harm to multiple organizations simultaneously.
He believes that when planning and deploying AI, businesses must recognize aggregation risks, particularly when using the underlying model for decisions that affect consumers.
Michael argues that small, task-specific models are better in terms of risk, as the intended use cases are clearly defined. Creating small models makes testing easier, error rate measurements easier, and unpredictable performance shifts are likely to occur. In contrast, very large models may behave inconsistently in different use cases, with low error rates in one scenario, while others may have very high rates.
He gives an example of the 2023 GPT-4 update. In models with error rates below 5% for certain tasks, those error rates suddenly occurred after retraining to exceed 90%. The differences, he says, highlight the vulnerability of the larger general model.
To address the issue, Michael recommends that companies consider using different underlying models or deliberately choosing a slightly weaker model architecture if similar tasks do not have much to do with those used elsewhere in the organization. Filling his arguments on this issue, he emphasizes that diversification helps deliver appropriate performance while reducing aggregation risk.
