At Ward and Smith’s Annual In-House Counsel Seminar, cybersecurity, data privacy and technology lawyer Mayuk Sirkar shared comprehensive guidance on the strategic role of artificial intelligence (AI) in the modern business environment, the key risks associated with its implementation, the evolution of AI regulation, and strategies for AI governance. Read Part 1 of this report here.
Developing an AI governance process
The first step in the AI governance process is to perform a comprehensive inventory of existing AI systems and use cases. This includes:
- Determine the purpose of your AI tools and understand whether they will serve internal operations or customers
- Partner with IT and procurement to conduct AI audits and catalog all AI systems, including shadow IT.
- Facilitate workshops to identify AI integration opportunities, inviting leaders from multiple layers of your organization to brainstorm, provide feedback, and problem-solve.
- Deploy departmental surveys to uncover repetitive, data-intensive or decision-intensive tasks that could benefit from AI, such as resume screening or fraud detection.
- Map use cases to legitimate business purposes to meet data privacy requirements
Appropriate governance according to risk
It’s important to find the right tool for your business needs. “The key here is to apply proportionate governance,” Sarkar said. “You have to consider how much data a particular tool requires. If there’s another tool that accomplishes the same goal with less data, it might be a better way.”
Balancing risk and business objectives is an ongoing challenge for legal departments, and the use of AI is no exception. Resume screening and loan applications are tools that should be classified as high risk because they affect legal and material rights.
“High-risk AI tools require rigorous monitoring and formal impact assessment,” Sarkar explained. “In some cases, it may be prohibited.”
Tools that create external marketing copy and internal analysis reports pose a moderate risk to your organization. Output should always be reviewed by a human.
Use cases for low-risk AI tools might include brainstorming and summarizing published articles. “These are likely to be governed by general usage policies that are already in place,” Sarkar said. “The key here is to go beyond functionality and evaluate the tool against legal and operational checklists to see if it fits your use case and risks. Basically, it’s about deciding whether it’s worth squeezing the juice out of it.”
Questions that legal teams should consider in this context include the level of security maintained by the vendor and whether the data is encrypted. “When you hire a vendor, ideally you want zero data retention, including a clear statement that the data will be deleted after use.Of course, if the data is not in the vendor’s systems, it cannot be stolen in the event of a breach,” Sarkar added.
We recommend that you limit your data usage. “You’ll probably want a private instance option to prevent third parties or the public from accessing your data,” Sircar said.
Accuracy and reliability are important metrics. “This refers to the rate of known errors, illusions, and biases,” Sarkar explained. “You will want to ask what your vendor is doing to resolve these issues, or your organization may be subject to a negligence claim.”
Vendors must be able to demonstrate compliance with key regulations. For example, some AI tools only comply with U.S. regulatory requirements and may not be approved for international use.
Ask the right questions about compliance and privacy
With the idea that setting up a due diligence framework is paramount, organizations should create a standardized questionnaire for AI vendors. “This is a very thorough process that combines technical security and ethical review. This should be a mandatory aspect of procurement,” Sarkar advised.
The issue of data provenance needs to be addressed. “Where does the training data come from? Is it legally licensed? Is proprietary data involved? These are important questions to consider,” Sarkar said.
Vendors should be able to explain in general terms how the AI reaches its conclusions. Similarly, vendors should be able to provide transparency and explain how their tools work.
“Ask your vendor what steps have been taken to test and reduce bias,” Sircar added. “Ask for a copy of the fairness audit, so you can see if the vendor is trying to address the issue or just turning a blind eye.”
Organizations should consider vendor security protocols such as incident response plans, access controls, and safeguards. Having audit privileges is an important way to ensure that vendors are complying with contractual security and privacy requirements.
“Check whether the vendor has the right to audit service providers and sub-processors,” Sarkar pointed out. “You should be able to provide a summary of the audit, or at least demonstrate that the audit meets compliance requirements.”
You need to consider the lineage of your data and how it is retained. “Work with your IT team to understand how prompts, uploads, outputs, metadata, telemetry, etc. are handled,” Sircar advised.
Transparency and safety controls must be factored into the equation. Organizations should maintain the option to turn off high-risk features to protect sensitive and confidential information.
Mandatory pilot testing is essential to test tools against your use case and data. Running a small pilot with real data can help identify potential issues and vulnerabilities.
“If a vendor is reluctant to offer a pilot, that’s a big red flag. Our general recommendation is to not procure enterprise AI tools without running a pilot,” Sircar explained.
Sircar then showed the audience a screenshot of the company’s AI tool comparison spreadsheet. He said assessment tools should be customized to an organization’s realities and that anyone should feel free to get in touch if they would like to discuss what should be included in an organization’s spreadsheets.
AI contract negotiation
“This is where we as lawyers can add immense value – don’t take vendor documentation at face value and ensure that AI risks are explicitly and contractually allocated,” Sarkar added.
Creating standard, non-negotiable AI contract addenda is an effective way to address concerns and reduce risk. The addendum must include data use restrictions that prohibit the vendor and its subprocessors from using customer data to train, develop, or improve AI models without explicit written consent.
Documentation must clearly define ownership of intellectual property. The contract should specify that the organization owns the generated prompts and output to the fullest extent permitted by law.
Broad coverage should be included. “Vendors need to proactively support their products with intellectual property coverage, data breach, bias and psychedelic correction negligence coverage. We want our providers to indemnify us in case something goes wrong on their end,” Sarkar said.
Compliance assurance ensures that the vendor’s systems comply with applicable laws, including privacy laws. “There may be some backlash here,” Sarkar said. “Vendors may say that data sent to AI cannot be deleted.”
Security and audit rights should be negotiated to include topics such as vulnerability management, breach notification timelines, and independent certification reporting. “There will need to be meaningful audit powers available as needed,” Sarkar advised.
Regarding transparency and change control, the appendix should include language regarding model limitations, notifications, and review of significant updates and changes. Additionally, organizations should retain the right to disable high-risk features, object to changes, and terminate contracts.
Practical policy for the imperfect
Developing an internal AI governance policy is your first line of defense. “There’s no need to reinvent the wheel, so consider integrating AI rules into the policies you already have in place, such as acceptable use policies, data classification policies, and information security policies,” Sircar explains.
Policy updates should include clear prohibitions against inputting personally identifiable information, trade secrets, or privileged data into public AI tools. For example, since AI glasses integrate with social media, it may be worthwhile to incorporate a policy prohibiting the use of AI glasses into your existing social media policy.
Employees should receive training on AI risks and compliance. “You need to understand that there is a critical difference between free public tools and enterprise-grade solutions,” Sarkar points out.
Human validation should be mandatory for any substantive work generated by AI. Governance policies should include disclosure rules that provide clear guidance on when employees must disclose that content was created by AI.
A cross-functional team should be established for AI governance. This committee is tasked with policy oversight and must be able to adapt to regulatory and operational changes.
“Legal departments can transform their role from a gatekeeper to a strategic enabler. With a robust governance framework, they can deploy AI sustainably, defensively, and strategically, turning a potential liability into a competitive advantage,” Sarkar concluded.
Q&A
In response to questions from the audience, Sarkar said legal teams need to be as transparent as possible and exercise their audit powers when necessary. Audit privileges should include the right to evaluate incidents and security issues.
Another audience member asked whether the law is keeping up with the rapid pace of agentic AI innovation. “The law is moving at a glacial pace and technology will always outpace the law,” Sarkar said. “However, data privacy rules are being introduced fairly rapidly and the situation is changing weekly.”
Although data processing agreements are mandatory and serve as a baseline, these agreements are generally not sufficient to protect confidential information or copyright.
The output is not copyrighted. However, you may be able to copyright your custom-trained AI models.
If a service provider is reluctant to provide coverage for bias or illusions, companies should consider finding another provider.
A digital omnibus could force U.S. companies to comply with EU AI regulations. However, some recent changes may lower regulatory hurdles for small businesses.
If a vendor refuses to negotiate the use of data for training, organizations should analyze the risks and develop a governance plan to limit the inclusion of sensitive information.
