UNIVERSITY PARK, Pa. — Shagufta Menaz, assistant professor of computer science and engineering in the Penn State College of Electrical Engineering and Computer Science, has been awarded a five-year, $632,430 National Science Foundation (NSF) Early Career Development (CAREER) Award for her project, “Privacy Audit Framework and Defense for Machine Learning Models Trained on Tabular Data.”
Mehnaz explained the goals of the project in this Q&A.
Q: What do you want to understand or solve through this project?
Menaz: The goal of this project is to better understand and prevent privacy risks that arise when machine learning (ML) systems are trained on sensitive personal data, such as medical or financial records. Specifically, we aim to study a type of privacy violation called a model inversion attack. In this attack, an attacker can infer private details about an individual by strategically querying a trained ML model.
Although such attacks have been widely studied in image-based ML systems, there is little understanding of how they affect the more common tabular data used in real-world applications. Tabular data refers to structured data sets, such as databases, that are organized in rows and columns. Each row represents a separate record, such as a patient, customer, or transaction, and each column corresponds to a specific attribute, such as age, income, diagnosis, or account balance.
This project aims to fill the knowledge gap by developing a new framework to systematically audit ML models for privacy risks, identify which individuals or groups are most vulnerable, and design robust defenses to mitigate these risks. Ultimately, we aim to enable the safe and fair use of ML models without compromising the privacy of the individuals who own the data that makes them possible.
Q: What impact could advances in this field have on society?
Menaz: Advances in this effort will strengthen public trust in artificial intelligence (AI) systems by increasing their transparency, accountability, and privacy protections. With the increasing use of ML models in sensitive fields such as healthcare, education, and finance, preventing privacy breaches is critical to protecting individuals' personal data and maintaining trust in data-driven technologies.
This project provides practical tools for auditing ML privacy risks and a public dashboard to track known vulnerabilities. The project will also help promote fairness and equity in the use of AI by identifying and addressing disparities in privacy risks where certain groups may face higher risks. In the long term, these achievements will help ensure that the societal benefits of machine learning are achieved in a responsible and ethical manner.
Q: Will undergraduate or graduate students contribute to this research? How?
Menaz: Yes, both undergraduate and graduate students will play important roles in the research and teaching components of this project. Graduate students will conduct in-depth research into machine learning vulnerabilities, develop new auditing algorithms, and design privacy-preserving defenses that balance data protection and model performance. Undergraduate students engage in collaborative research, capstone projects, and hands-on data analysis tasks.
In addition, students will participate in the creation of open source tools and educational materials that can be used more widely. Competitions and workshops planned for this project will provide further experiential learning opportunities and inspire students to pursue careers in reliable and secure AI research.
Q: NSF CAREER awards not only fund research projects, but also recognize recipients' potential as researchers, educators, and leaders in their fields. How do you want to realize that potential?
Menaz: This award enables you to develop your potential as researchers, educators, and leaders by integrating rigorous scientific research with innovative teaching and mentoring. As a researcher, I aim to establish a leading machine learning privacy and fairness program and develop a systematic framework for understanding, measuring, and mitigating privacy risks in real-world datasets. My job is to push the boundaries of knowledge in this field and create tools and methodologies that can be widely adopted. As an educator, I plan to design and teach both undergraduate and graduate courses on security and privacy in ML, and publish materials to expand access to these emerging topics. As a leader, I aim to foster and coordinate research collaboration across institutions and disciplines, and foster a community that advances privacy-aware machine learning. Through competitions, public dashboards, and advocacy, I will help set research standards, disseminate best practices, and shape the broader research agenda in this important field.
We are deeply grateful to NSF for this CAREER award for providing the continued support necessary to build a long-term, integrated research and education program. This stability allows me to pursue a consistent vision. This means training students to think critically about data ethics and security while advancing the theory and practice of privacy auditing in machine learning.
This award will help establish a dedicated research group and develop publicly available tools that will benefit both academia and industry. It will also enable us to expand our educational and outreach efforts to inspire young students and broaden their participation in computing. We are grateful for the support of NSF. This provides us with the foundation we need to become a leader in the development of responsible, privacy-friendly AI technologies that protect individuals and strengthen public trust.
At Penn State, researchers solve real-world problems that impact the health, safety, and quality of life of people across the Commonwealth, the United States, and around the world.
For decades, federal support for research has fostered innovation that makes our nation safer, our industries more competitive, and our economy stronger. Recent federal funding cuts threaten this progress.
For more information on how federal funding cuts will affect our future, please visit Research or Regress.
