At Apple, we believe privacy is a fundamental human right. As AI improves in capabilities and becomes more integrated into people’s daily lives, it is increasingly important to advance research into privacy protection techniques to ensure users protect their privacy while enjoying innovative AI experiences.
Apple’s fundamental research consistently drives the cutting edge of the field, and we hosted a workshop on privacy-preserving machine learning and AI earlier this year. This two-day event brought together Apple researchers and members of the broader research community to discuss the latest in privacy-preserving ML and AI, focusing on three key areas: private learning and statistics, underlying models and privacy, and attacks and security.
Workshop presentations and discussions explored advances and open questions in privacy and ML, including federated learning, statistical learning, trust models, attacks, privacy accounting, and the unique challenges posed by underlying models. These research areas build on innovations in rigorous privacy and security assessments and bridge theoretical frameworks with real-world applications.
In this post, we will share recordings of selected talks and summaries of publications discussed at the workshop.
Adaptive methods are preferable in high-privacy settings: SDE perspective: Enea Monzio Compagnoni (University of Basel), Alessandro Stanghellini (University of Basel), Rustem Islamov (University of Basel), Aurelien Lucchi (University of Basel), and Anastasiia Koloskova (University of Zurich)
Capture with Captions: Memory and Its Relaxation in Clip Models, Authors: Wenhao Wang (CISPA), Adam Dziedzic (CISPA), Grace C. Kim (Georgia Institute of Technology), Michael Backes (CISPA), and Franziska Boenisch (CISPA)
Apple researchers combine machine learning and homomorphic encryption in the Apple ecosystem
“Concurrent Configuration for Differentially Private Continuous Mechanisms” Monika Henzinger (Austrian Institute of Science and Technology), Roodabeh Safavi (Austrian Institute of Science and Technology), Salil Vadhan (Harvard University)
Contextual Agent Security: Policies for Any Purpose, by Lillian Tsai (Google) and Eugene Bagdasarian (Google)
Less stuffing, more learning: Training data pruning improves fact recall by Jiayuan Ye, Vitaly Feldman, Kunal Talwar
Unraveling foreground and background memory in diffusion models by Jimmy Z. Di (University of Waterloo), Yiwei Lu (University of Ottawa), Yaoliang Yu (University of Waterloo), Gautam Kamith (University of Waterloo), Adam Dziedzic (CISPA), and Franziska Boenisch (CISPA)
Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs by Xun Wang (CISPA), Jing Xu (CISPA), Franziska Boenisch (CISPA), Michael Backes (CISPA), Christopher A. Choquette-Choo (Google DeepMind), Adam Dziedzic (CISPA)
Efficient Privacy Loss Accounting Considering Subsampling and Random Assignment, by Vitaly Feldman and Moshe Shenfeld (Hebrew University of Jerusalem, research conducted while at Apple)
Taking my eyes off my data: Exploring differential private federated statistics to support algorithmic bias assessment across demographic groups with AI staff partnerships
Finding NeMo: Localization of neurons responsible for memory in a diffusion model by Dominik Hintersdorf (German Research Center for Artificial Intelligence (DFKI), Darmstadt University of Technology), Lukas Struppek (German Research Center for Artificial Intelligence (DFKI), Darmstadt University of Technology), Kristian Kersting (German Research Center for Artificial Intelligence Intelligence (DFKI), Darmstadt University of Technology, Hessian AI Center), Adam Dziedzic (CISPA), Franziska Boenisch (CISPA)
Stochastic Parrot Flock: Differential Private Prompt Learning of Large-Scale Language Models by Haonan Duan (University of Toronto and Vector Institute), Adam Dziedzic (University of Toronto and Vector Institute), Nicolas Papernot (University of Toronto and Vector Institute), Franziska Boenisch (University of Toronto and Vector Institute)
Differential privacy on local nodes: Sofya Raskhodnikova (Boston University), Adam Smith (Boston University), Connor Wagaman (Boston University), Anatoly Zavyalov (Boston University)
Memorization in self-supervised learning improves downstream generalization by Wenhao Wang (CISPA), Muhammad Ahmad Kaleem (University of Toronto and Vector Institute), Adam Dziedzic (CISPA), Michael Backes (CISPA), Nicolas Papernot (University of Toronto and Vector Institute), and Franziska Boenisch (CISPA)
Congzheng Song and Xinyu Tang, Memory-Efficient Backpropagation for Fine-Tuning LLM on Resource-Constrained Mobile Devices
Open LLMs are necessary for today’s private adaptations and outperform closed alternatives by Vincent Hanke, Tom Blanchard, Franziska Boenisch, Iyiola E. Olatunji, Michael Backes, Adam Dziedzic (CISPA)
Piquantε: Private quantile estimation in a two-server model by Hannah Keller (Aarhus University), Jacob Imola (BARC, University of Copenhagen), Rasmus Pagh (BARC, University of Copenhagen), Fabrizio Boninsegna (University of Padova), Amrita Roy Chowdhury (University of Michigan)
Privacy Inference in Ambiguous Contexts by Ren Yi (Google Research), Octavian Suciu (Google Research), Adrià Gascón (Google Research), Sarah Meiklejohn (Google), Eugene Bagdasarian (Google Research), Marco Gruteser (Google Research)
Ravan: Multihead Low-Rank Adaptation for Federation Fine-Tuning, by Arian Raje (CMU), Baris Askin (CMU), Divyansh Jhunjhunwala (CMU), and Gauri Joshi (CMU)
Robin Hood and Matthew effects: Differential privacy has a different impact on synthetic data by Georgi Ganev (University College London, Heisei), Bristena Oprisanu (University College London), and Emiliano De Cristofaro (University College London)
Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies Authors: Mason Nakamura (University of Massachusetts Amherst), Abhinav Kumar (University of Massachusetts Amherst), Saaduddin Mahmud (University of Massachusetts Amherst), Sahar Abdelnabi (ELLIS Institute Tübingen, Intelligent Systems MPI, Tübingen Center for AI), Shlomo Zilberstein (University of Massachusetts Amherst), and Eugene Bagdasarian (University of Massachusetts Amherst)
The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Data Georgi Ganev (UCL, SAS), Meenatchi Sundaram Muthu Selva Annamalai (UCL), Sofiane Mahiou (SAS), Emiliano De Cristofaro (UC Riverside)
The Inadequacy of Similarity-Based Privacy Metrics: Privacy Attacks on “True Anonymous” Synthetic Datasets, by Georgi Ganev (UCL, SAS) and Emiliano De Cristofaro (UC Riverside)
“Data storage tradeoffs due to strong data processing inequalities” Vitaly Feldman, Guy Kornowski (Weizmann Institute of Science; research done while at Apple), and Xin Lyu (UC Berkeley; research done while at Apple)
Many people contributed to this workshop, including Vitaly Feldman, Christina Ilvento, Tuki Koga, Audra McMillan, Congzheng Song, Kunal Talwar, Andreas Thoma, and Jiayuan Ye.
