The UK’s National Cyber Security Center has published a paper on Adversarial Attacks against Machine Learning and AI, setting out a framework for understanding attacks that target the manipulation of ML models. This document introduces a common language aimed at supporting AI security awareness, threat modeling, and collaboration.
According to the NCSC, ML systems have a larger attack surface than traditional software due to their faster development cycles, proprietary architectures, large model sizes, and extensive use of open source components. Distinguish adversarial machine learning attacks from a broader range of cyberattacks by focusing on attacks that exploit vulnerabilities specific to the architecture, training, or operation of ML models.
This document defines seven attack classes:
- Model characterization
- model inversion
- training data poisoning
- Training a malicious model
- Model input operations
- Manipulating model artifacts
- Modeling hardware attacks
These attacks can occur throughout development, training, and deployment, and can target both hardware and software components.
The NCSC also maps these attack classes against eight potential goals for malicious actors, including reconnaissance, degraded performance, wasted resources, embedded covert behavior, evaded detection, exfiltrated data, and gained broader system access. The table on pages 11-12 associates each class with one or more goals.
The paper argues that standard cybersecurity controls remain fundamental, but notes that ML-specific weaknesses often require specialized mitigations that are not yet mature or widely deployed.
It calls for further research into untapped areas such as model hardware attacks and malicious model training, and recommends greater use of frameworks and guidance from the NCSC, ETSI and the UK Government’s AI Cybersecurity Code of Practice.
Want to learn more about AI, technology, and digital diplomacy? Then contact Diplo chatbot.
