As AI spreads like wildfire into the business world, the data it harnesses is becoming more and more important to enterprises.
From privacy and security to ethical issues and training bias, the range of data-related AI considerations is wide and growing. For many companies, this means rethinking their policies and practices, even if they haven’t formally adopted AI technology within their organizations.
At the Fortune Brainstorm Tech Conference in Deer Valley, Utah this week, Check Point chief technology officer Dorit Dole said how much internal data is already being used for AI tools within their own organizations. Companies said they didn’t understand how they were being used. “Their data and information are already out there,” she said.
Dor joined executives from PagerDuty, Salesforce, and Signal to share their perspectives and experiences on the subject of data in the age of AI in group panel discussions at the conference.
Employees are experimenting with tools like ChatGPT, feeding internal data into AI tools. This means businesses face a significant problem of data breaches that can compromise both their own competitive information and private customer data. Clara Shih, CEO of the Salesforce AI business, said some of the AI tools used by employees at various companies “traditionally don’t have the clean separation that companies would expect from a secure database.” said.
Shih explained how the large language models that power generative AI tools require as much context as possible from the user in order to produce the most relevant and accurate responses. “If you’re not careful how you build it, by default the context you give to the prompt will be learned by the model itself.”
Sean Scott, Chief Product Development Officer at PagerDuty, expressed similar concerns, but said it all had to do with following security best practices.
It starts with “What are your policies?” what is your crown jewel? What data do you want to protect? I was.
Protection from mysterious model data
Protecting internal data from exposure is important, but companies must also address the quality of the external data they ingest. Signal president Meredith Whittaker said most of his off-the-shelf AI large language models are black boxes. “They know what the data is, but you don’t know what the data is,” she said.
Companies that want to introduce these AI tools into their operations run the risk of inaccurate or unpleasant results due to mysterious data.
“What we can do, then, is move the model into a shape that fits the domain, make it a shape that is more specialized, and use other data that might make it less offensive. It’s all about fine-tuning,” Whittaker said. “But I think we need to clarify the lack of agency on these questions.”
Whittaker, an adviser to the US FTC, has called for more regulation to block data that has trouble flowing and limit data entering the “bloodstream.”
Check Point’s Doe warned that regulation is just the beginning. “Regulations only raise the minimum requirements and never really take you to a safe place,” Doe said.
Do, meanwhile, said much of the burden of data processing in the AI age falls on the shoulders of already overburdened corporate chief information security officers.
“CISOs, who used to be exhausted, are now suddenly given this mandate with many unknowns, including legal aspects.”
