As companies accelerate adoption of GenAI ahead of the full implementation of the DPDP, Malcolm Gomez, chief operating officer at digital identity verification company IDfy, believes that uncontrolled AI use and “shadow AI” will pose increased privacy risks.
“The next big privacy breach may not start with a hacker. It may start with an employee pasting customer data into a public AI tool to get work done faster. The real risk is not the AI itself, but the uncontrolled use of AI,” Gomez said.
The warning comes as businesses rapidly deploy generative AI across customer onboarding, fraud detection, identity verification, and business operations, while preparing for full implementation of the Digital Personal Data Protection (DPDP) Act by 2027.
This shift highlights the growing challenge for Indian companies to balance AI-driven innovation with privacy and regulatory compliance. Industry experts say organizations are moving beyond traditional concerns about consent collection to broader questions about how data is used, stored and managed throughout the AI lifecycle.
According to Gomes, generative AI is outpacing enterprise governance frameworks, creating blind spots on where customer data is shared and how it is subsequently processed by third-party AI platforms.
When personally identifiable information (PII) enters an uncontrolled AI environment, companies often lose visibility into where that data resides, how long it is retained, whether it is used to train models, and how it can be deleted or recovered later, he said.
This concern is not just theoretical. Citing industry estimates, Gomez said about 15% of employees enter sensitive information into public large-scale language models (LLMs) and about 40% of organizations report at least one AI-related privacy incident.
The increasing use of what experts call “shadow AI” is emerging as a major governance challenge. Similar to shadow IT, employees have access to AI tools without formal approval from procurement or IT departments and often use them for seemingly innocuous tasks such as summarizing documents, translating content, and generating responses.
“Shadow AI is the new shadow IT. The difference is that sensitive customer data can be pulled out of the enterprise management environment in a single browser tab,” said Gomez.
He pointed out that existing cybersecurity controls such as firewalls, endpoint protection, and network security are not designed to handle situations where employees voluntarily share data with external AI platforms.
As a result, companies must implement new safeguards such as immediate levels of monitoring, data classification systems, access controls, AI usage policies, and privacy-by-design workflows.
The rise of GenAI is forcing organizations to rethink their approach to DPDP compliance. Traditional privacy programs are built around structured data flows involving databases, applications, and known vendors. AI introduces additional layers such as prompts, chatbot logs, search systems, AI agents, and model outputs that cannot be collected with traditional data inventories.
For organizations preparing for DPDP compliance, key questions now include whether personal data used in AI systems is consistent with the purpose for which consent was originally obtained, whether deletion requests can be honored once the data enters the AI workflow, and whether adequate audit trails exist to prove compliance.
Gomez argued that DPDP preparation is no longer just a legal or compliance exercise.
“Organizations that get this right will not treat AI governance and privacy compliance as separate programs; they will build them together,” he said.
Industry observers increasingly view privacy-first AI as a business differentiator rather than a regulatory mandate. As boards and regulators pay closer attention to AI-related risks, digital trust is emerging as a strategic priority alongside innovation.
According to IDfy, for companies to scale AI responsibly, they will need three foundational layers: an AI governance framework that defines approved tools and use cases, consent orchestration mechanisms to ensure lawful and purposeful data use, and internal data controls that cover masking, redacting, access restrictions, and audit logging.
India’s privacy regime is maturing with the acceleration of AI adoption, and organizations that can demonstrate transparency, traceability, and verifiable control over how personal data is used by AI systems will be well-positioned to gain customer trust and avoid regulatory scrutiny.
“The future of privacy governance is not about slowing down AI; it’s about creating a layer of control that allows AI to scale securely,” Gomez said.
