The US National Institute of Standards and Technology (NIST) has released a new open-source software tool for testing the resistance of machine learning (ML) models to various types of attacks.
The tool, called Dioptra, was released Friday along with new AI guidance from NIST to mark the 270th anniversary.Number It has been one day since President Joe Biden signed the Executive Order on the Safe, Secure, and Responsible Development of AI.
The Dioptra tool, available on GitHub, meets the Executive Order's requirement that NIST assist in testing AI models and also supports the “measure” feature of NIST's AI Risk Management Framework.
“Open source development of Dioptra will begin in 2022, but it was in an alpha 'pre-release' state until last Friday, July 26,” a NIST spokesperson told SC Media. “Major new features from the alpha release include a new web-based front end, user authentication, and provenance tracking of all elements of an experiment, allowing for reproducibility and validation of results.”
Free Dioptra AI Testing Platform Measures Impact of Three Attack Categories
Previous NIST studies have classified attacks against machine learning algorithms into three main categories: evasion, poisoning, and oracles.
According to NIST, evasion attacks aim to trigger inaccurate model responses by manipulating data inputs (e.g., adding noise), poisoning attacks aim to modify training data to reduce a model's accuracy and lead to erroneous associations, and oracle attacks aim to “reverse engineer” a model to obtain information about the training dataset or parameters.
The Dioptra tool was originally built to measure attacks against image classification models, but can also be adapted to test other ML applications, such as speech recognition models.
This free platform allows users to determine the extent to which the three categories of attacks mentioned above affect their model's performance, as well as evaluate the use of different defensive measures such as data sanitization and more robust training methods.
The open-source testbed has a modular design to support experimentation with different combinations of models, training datasets, attack tactics, defenses, and more.
The interactive web interface accommodates a variety of user skill levels.
The newly released Dioptra version 1.0.0 includes a number of features to maximize accessibility for first-party model developers, second-party model users or buyers, third-party model testers or auditors, and researchers in the ML field.
Dioptra 1.0.0 features a modular architectural design and a user-friendly web interface, as well as extensibility and interoperability with Python plugins that add functionality. Dioptra also comes with documentation and demos to help users with little programming experience get comfortable experimenting with Dioptra.
Dioptra tracks experiment history, including inputs and resource snapshots supporting traceable and reproducible testing, and can uncover insights that lead to more effective model development and defense.
The tool can be deployed in a multi-tenant environment to facilitate sharing of resources and components among users, but can also be deployed on a single local machine.
Dioptra is most compatible with Unix-based operating systems such as Linux and MacOS, and experiments typically require significant computational resources. The Dioptra architecture has been formally tested on an NVIDIA DGX server equipped with four graphics processing units (GPUs).
“User feedback helped inform the design of Dioptra, and NIST will continue to gather feedback and refine the tool,” a NIST spokesperson told SC Media.
NIST Releases New Guidance to Advance AI Safety Goals
The release of the Dioptra software package coincided with the publication of a new draft document on Friday from NIST's AI Security Laboratory, which focuses on managing the risks of “dual-use” infrastructure models that could be used for both good and bad purposes.
NIST will accept public comments on the guidance document until September 9th.
In addition, NIST has published three final guidance documents that were previously available as draft documents.
The first document highlights 12 unique risks of generative AI and over 200 recommended actions to help manage these risks, the second outlines “Secure Software Development Practices for Generative AI and Dual-Use Foundation Models,” and the third lays out a plan for global collaboration on developing AI standards.
“While generative AI offers potentially transformative benefits, it also poses risks that are very different from those seen with traditional software. These guidance documents and testing platforms will inform software developers of these unique risks and help them develop ways to mitigate them while supporting innovation,” NIST Director Laurie E. LoCascio said in a statement.
