Stress testing AI vision systems: Rethinking how we generate adversarial images

Deep neural networks (DNNs) are the cornerstone of modern AI technology and are driving a thriving research field for image-related tasks. These systems have applications in medical diagnostics, automated data processing, computer vision, and various forms of industrial automation, to name a few. As reliance on AI models increases, so does the need to thoroughly test them using adversarial examples. Simply put, an adversarial example is an image that has been strategically altered with noise to trick the AI ​​into making a mistake. Understanding adversarial image generation techniques is essential for identifying vulnerabilities in DNNs and developing more secure and reliable systems.

Despite its importance, current techniques for creating adversarial examples have significant limitations. Scientists mainly L_p-standard. This keeps the changes subtle, but often results in grainy artifacts that look unnatural because they don’t match the texture of the original image. As a result, even if the noise is small and difficult to see, it can be easily detected and blocked by security prefilters that look for anomalous frequency patterns. Therefore, a notable challenge in this field is not just to minimize the amount of noise, but to create more sophisticated adversarial attacks.

Against this background, doctoral student Masatomo Yoshida and professor Masahiro Okuda of Doshisha University’s Graduate School of Science and Engineering developed a method to match additive noise in adversarial examples to the “spectral shape” of the image. Their research was published in the journal Volume 13 IEEE access On December 24, 2025, an innovative framework called Input-Frequency Adaptive Adversarial Perturbation (IFAP) was introduced.

Unlike previous frequency-aware techniques that operate only on specific frequency bands, IFAP uses a new spectral envelope constraint. This adaptively matches the added noise to the entire frequency distribution of the input image, ensuring that the perturbation is spectrally faithful to the original content.

The researchers tested IFAP across a variety of datasets, including house numbers, common objects, and complex textures such as terrain and fabrics. To evaluate its performance, they used a comprehensive set of metrics, including a new metric they developed called frequency cosine similarity (Freq_Cossim). While standard metrics typically check for pixel-level errors, Freq_Cossim specifically measures how well the shape of the noise’s spectral profile frequencies matches the spectral profile of the original image.

Results showed that IFAP significantly outperforms existing adversarial generation techniques in structural and textural similarity to the source material. Despite becoming more visually natural and sophisticated, adversarial attacks remained highly effective and successfully fooled a wide range of AI architectures. Interestingly, the researchers also demonstrated that these harmonized perturbations were more resistant to common image cleaning techniques such as JPEG compression and blurring. Because noise is often built into the natural texture of an image, it is very difficult to remove it with simple transformations without significantly changing the image itself.

IFAP has important implications for how adversarial examples are used in AI research. Understanding how to generate noise that matches human perception will allow researchers to implement better adversarial attacks, conduct stress tests, and retrain AI models to become more robust. “We believe that in fields such as medical diagnosis, this may lead to the development of highly reliable AI models that are not affected by slight changes in image quality or noise.” says Professor Okuda.

Looking ahead, this study sets new benchmarks for how to evaluate the safety and performance of AI in image-centric tasks. “Evaluation criteria that emphasize consistency with human perception and frequency characteristics, such as those proposed by our research, are likely to become more common in the next 5 to 10 years.Professor Okuda concluded as follows.This change is expected to increase the reliability of AI systems that support important social infrastructure such as medical care and transportation.”

About Masatomo Yoshida Doshisha university, japan
Masatomo Yoshida received his master’s degree from Doshisha University, Japan in 2021 and his master’s degree in 2023. He is currently pursuing a Ph.D. Completed doctoral course at Doshisha University Graduate School of Science and Engineering. He is also a Japan Society for the Promotion of Science (JSPS) Special Research Fellow and received the JST Spring Scholarship (support for pioneering research by the next generation). His research interests include analysis of spatiotemporal time series data, image processing, deep learning, and adversarial examples.

About Masahiro Okuda of Doshisha University
Masahiro Okuda (IEEE Senior Member) received his Bachelor of Arts, Master’s, and Ph.D. He received degrees from Keio University (Yokohama) in 1993, 1995, and 1998, respectively. From 1996 to 2000, he served as a special research fellow of the Japan Society for the Promotion of Science. He was a visiting scholar at the University of California, Santa Barbara, Santa Barbara, California, USA, and Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, in 1998 and 1999, respectively. From 2000 to 2020, he worked at the Department of Environmental Engineering, Kitakyushu City University, Kitakyushu City. Currently a professor at the Faculty of Science and Engineering, Doshisha University. His research interests include image restoration, high dynamic range imaging, multiple image fusion, and digital filter design. He received the SIP Distinguished Service Award in 2013, the IE Award in 2017, and the Contribution Award from the Institute of Electronics, Information and Communication Engineers.

Funding information
This research was supported in part by Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (25KJ2207 and 23K11174) and in part by Japan Science and Technology Agency (JST) Support for Next Generation Pioneering Research (SPRING) grant JPMJSP2129.

Media contact:
Research and Development Promotion Organization
Doshisha University
Kyotanabe City, Kyoto Prefecture 610-0394
E-mail: jt-ura@mail.doshisha.ac.jp