Towards fairer AI: Strategies for per-instance unlearning without retraining

The increasing reliance on machine learning models in critical applications has raised concerns about tampering and misuse. Once trained on a dataset, these models often retain information indefinitely, making them vulnerable to privacy violations, adversarial attacks, or unintended bias. Therefore, there is an urgent need for techniques that allow models to unlearn specific data subsets to reduce the risk of unauthorized access and misuse. Machine unlearning addresses this challenge by allowing pre-trained models to be modified to forget certain information, making them more resilient to potential risks and vulnerabilities.

Machine learning unwinding is intended to modify a pre-trained model so that it forgets a specific subset of data. Initially, this technique focused on shallow models such as linear regression and random forests, removing unnecessary data while maintaining performance. In his recent work, he extends this to deep neural networks, and primarily he uses two approaches. One is per-class, which forgets the entire class and preserves the performance of other classes, and the other is per-instance, which targets individual data points. However, traditional methods aimed at inducing models to retrain without unnecessary data have proven ineffective against data leakage due to deep network interpolation functions.

A recent publication by a team of researchers from LG, New York University, Seoul National University, and the University of Illinois at Chicago highlights the limitations of existing methods, including the assumption of class-based unlearning settings and reliance on access to original training data. A new approach to overcome this was introduced. , information leakage could not be effectively prevented. In contrast, the proposed method introduces per-instance unlearning and pursues a more robust objective of preventing information leakage by ensuring that all data requested for deletion is misclassified. .

Specifically, the proposed framework defines the dataset and pre-trained model setup. The entire training dataset, denoted as Dtrain, is used to pre-train the classification model gθ: X → Y. The subset of data intended for unlearning is denoted as Df, and Dr represents the remaining dataset that needs to maintain prediction accuracy. This method works with only access to a pre-trained model gθ and an untrained dataset Df. Adversarial examples are very important in approaches that are generated by targeted PGD attacks and induce misclassification. The weight importance measure is calculated using the MAS algorithm to identify parameters that have a significant influence on the change in the output. These preparations prime the proposed framework, which consists of per-instance non-learning and regularization techniques to alleviate forgetting of remaining data.

This framework employs adversarial examples and weight importance measures for regularization. Adversarial examples help preserve class-specific knowledge and decision boundaries, while weight importance prevents forgetting by prioritizing important parameters. These two approaches improve performance, especially in difficult scenarios such as continuous unlearning, and provide an effective solution with minimal access requirements.

To evaluate the proposed non-learning method in the new method, the research team conducted experiments on CIFAR-10, CIFAR-100, ImageNet-1K, and UTKFace datasets and compared it with various baseline methods. . This new method leverages adversarial examples (ADV) and weighted importance (ADV+IMP) for regularization and demonstrated excellent performance in maintaining accuracy on residual and test data across different scenarios. The new method outperformed other methods even when continuously unlearning and modifying natural adversarial examples. Qualitative analysis demonstrated the robustness and effectiveness of the new method in maintaining decision boundaries and avoiding patterns of misclassification. These findings highlight the effectiveness and safety of the new unlearning method.

Please check paper. All credit for this study goes to the researchers of this project.Don't forget to follow us twitter.Please join us telegram channel, Discord channeland linkedin groupsHmm.

If you like what we do, you'll love Newsletter..

Don't forget to join us 41,000+ ML subreddits