How generative AI is changing the debate about responsibility

As regulators struggle to keep up with the pace of advances in generative AI (genAI), the issue of liability has taken center stage. AI-powered chatbots were thrust into the spotlight earlier this year when Air Canada was found liable and ordered to pay damages and legal costs after a chatbot gave passengers false advice.

Commenting on the “extraordinary” Air Canada case ruling, Helen Vaughn (pictured left), a partner at Clyde & Co., said companies are struggling to understand the new definition of liability. There are currently situations where the decisions at issue don't involve humans, and cases like this should prompt organizations to think carefully about how they use new technology with a full understanding of their own risk to the outcome, she said.

How can companies rethink how they approach liability risk?

It's clear that genAI represents a major shift in how companies approach responsibility, bringing new challenges and considerations to the fore. Key concerns are data management and intellectual property (IP). Generative AI systems often rely on vast amounts of data, increasing the risk of data privacy breaches and IP infringement.

This is a risk that requires strict data governance policies and robust cybersecurity measures to prevent unauthorized data use and ensure compliance with relevant regulations. “Don't get me wrong, we're still in the experimentation phase of this next generation of advanced AI tools,” says Rory Yates (pictured right), SVP of corporate strategy at EIS. “Unlike previous technology revolutions, this one is evolving and expanding more rapidly than anyone could have imagined, and liability has been a topic that has been overdue for consideration until now.”

Yates noted that ChatGPT hit 1 million users in five days and now has about 180 million users worldwide. “It's amazing,” he said. “If you pick up any industry paper, you'll always see a headline about ChatGPT being everywhere and everyone using it for a while already, and the insurance industry is no exception.”

Conversations about AI are counterproductive without considering that AI-generated tools may produce inaccurate or biased outputs. Companies that rely on AI-generated content or decisions need to ensure that these outputs are accurate and non-discriminatory. Particularly in sectors such as healthcare and finance, companies may face legal liability for negligence if they fail to verify the accuracy of AI-generated information or if a lack of transparency in AI systems leads to unexpected errors.

AI and Threat Actors – An Evolving Threat

Further complicating matters, the question of liability is not limited to inherent limitations or errors in the technology itself, but extends to how it is weaponized by threat actors. A key concern is how it could be used to create advanced deepfakes for malicious purposes, such as financial fraud, blackmail, and misinformation.

Additionally, the commoditization and democratization of digital technologies on the dark web has made sophisticated cyber tools more accessible to even the most inexperienced cybercriminals. The implications range from identity theft, fraud, and the potential for authentication system circumvention, creating significant risks for businesses that rely on digital verification processes.

Ultimately, it's about understanding the limitations of technology. “Historically, we've always understood the impact of human error,” Vaughn says. “Now we're dealing with technology that can produce outcomes we never thought of. And we have to ask whether that technology could be compromised by a bad actor and produce a completely different outcome.”

Overall, it is clear that as enterprises continue to adopt generative AI technologies, they must do so with a clear understanding of their security frameworks and how these can be leveraged to address emerging threats. There are several ways that enterprises can consider mitigating these risks, including implementing a comprehensive review process, providing proper training to employees who use AI tools, and establishing a clear accountability framework for AI-related decisions.

Yates emphasized that another important driver shifting the debate on liability is regulation. “Everyone knew regulation on AI was coming,” he said. “In the US, insurers Humana, Cigna and UnitedHealthcare are facing class action lawsuits from consumers and their heirs for deploying advanced technology to deny claims. It's important that they act responsibly.”

These codes and standards will help the industry chart a path to success by speeding up the development process and helping companies avoid the risk of compliance teams thwarting their plans or, even worse, regulators stepping in and killing their business.