Business and technology insight company Gartner predicts that by 2029, most privacy incidents will result from AI-generated inferences about a person rather than from direct exposure of personally identifiable information (PII).
As regulatory and cost pressures reduce the amount of personal data that organizations store, threat actors are gaining access to artificial intelligence to carry out inference-based attacks.
Advances in generative AI and machine learning have made it possible to extract sensitive attributes such as health status and behavioral patterns from seemingly innocuous anonymized or aggregated data.
Bart Willemsen, Analyst Vice President at Gartner, commented on this change: “While organizations have traditionally focused on protecting raw personal data, AI can now reconstruct deep personal insights without breaking traditional data management.
“Privacy risks increasingly emerge from what AI algorithms infer about individuals, rather than what data is directly exposed.”
“Inference attacks are particularly dangerous because they often evade traditional detection mechanisms,” Willemsen added.
“Individuals could be exposed through AI-generated conclusions rather than leaked records, compromising data integrity and creating privacy risks that are difficult to detect, explain, and mitigate.”
Recommended reading
With this in mind, Gartner predicts that by 2028, spending on data integrity protection will equal investment in data confidentiality as organizations address the risk of inaccurate, biased, or fraudulent AI-generated profiles.
The insights firm also recommended that chief information security officers and other privacy leaders take a series of steps to address these new inference-based privacy risks.
For example, strengthen cybersecurity against AI-powered threats by investing in advanced monitoring, anomaly detection, and scenario planning capabilities designed to identify indirect exploitation patterns and inference-based threats.
It was also proposed to implement techniques such as differential privacy, synthetic data, and privacy-aware machine learning to process data in a protected manner and employ privacy-enhancing techniques (PET) to reduce re-identification risks.
Additionally, in the move from publishing data to publishing insights, it was also advocated for incorporating AI governance into privacy programs, strengthening data minimization and lifecycle management, and promoting transparency and human oversight.
