In October, the UK NCSC’s 2025 Annual Review. CEO Richard Horne has warned that businesses’ futures are at risk if they are not prepared for cyberattacks. The urgency behind this statement is backed up by data. NCSC handled 204 major cyber incidents From September 2024 to 2025, and 43% of UK companies I reported a violation last year.
The review was immediately followed by an open letter from government ministers urging CEOs to “make cyber a board-level priority”. The message is clear. Cybersecurity is no longer optional, nor is it just for large companies. It is a strategic imperative for any organization.
Cyber threats are clearly escalating, with recent attacks on well-known companies such as Marks & Spencer, Co-op, Jaguar Land Rover (JLR) and Harrods exposing vulnerabilities in a variety of areas. JLR is estimated to have cost the UK economy more than £2bnespecially when considering supply chain disruptions. But the impact goes far beyond the balance sheet, with the lives of thousands of people affected.
For critical national infrastructure that supports public safety and economic stability, the consequences of a successful cyberattack can be devastating, and unfortunately, based on track record, it is not a matter of when.
Is AI a double-edged sword in cybersecurity?
The NCSC Annual Review also highlights the growing role of artificial intelligence (AI) in cybersecurity. Review guidance such as the AI Security Code of Practice, which focuses on ensuring the security of AI model development and deployment. However, AI is more than just a defensive tool; it is transforming both sides of the cyber battlefield. The ability to automate, scale, and adapt introduces new tactics and challenges, making it a powerful force multiplier for both attackers and defenders.
On the defensive side, AI is reshaping cybersecurity through advanced threat detection and automated response. Its use in vulnerability scanning and anomaly detection is rapidly expanding, and machine learning can help identify threats that traditional systems often miss. Microsoft’s Copilot and Purview are great examples of this change. Copilot integrates with security platforms to streamline threat analysis and automate incident response. Meanwhile, Purview powers data governance through AI-driven classification and monitoring. These tools provide real-time insight and rapid triage, which is critical for CNI operators who need to maintain uptime and safety.
However, important challenges remain. Fee. Expanding the hunting and correlation capabilities of AI tools increases operational costs.
How AI will impact the next wave of cyberattacks
Meanwhile, attackers are increasingly using AI to launch sophisticated evasion campaigns. Deepfake audio and video scams targeting executives are already occurring, and tools like Promptlock demonstrate how AI-generated prompts can automate lateral movement and privilege escalation. The speed and adaptability of AI may soon enable polymorphic malware that rewrites itself to evade detection. Hackers are now using smarter techniques to make their attacks more effective.
One method, reinforcement learning, can help coordinate and deliver harmful software in real time. At the same time, advanced malware like Emotet uses AI to probe a computer’s security and choose the best way to bypass it, making it extremely difficult for defenders to keep systems safe.
Companies like Anthropic are actively researching ways to make AI systems more resistant to adversarial manipulation. Their work on Constitutional AI and Red Team Large-Scale Language Models (LLMs) shows how attackers can exploit prompt injection and model behavior to produce harmful output or bypass safety measures. This highlights two aspects of AI: the same tools that increase productivity can be weaponized.
One of the most cited examples in Anthropic is the “Claude plays with Pokemon” experiment. Researchers embedded hidden instructions in seemingly harmless tasks that caused Claude to behave unintended. The goal was to test how easily the LLM agent can be hijacked or redirected without explicit malicious input. When applied to AI systems embedded in CNI environments, this type of manipulation can have devastating consequences, and even the slightest deviation can cause cascading failures.
Can AI plan ahead?
As AI-powered malware becomes more independent, experts question whether AI can plan ahead or change its goals. In March 2025, researchers at Anthropic discovered that the AI model Claude seemed to organize his thoughts before writing a poem, suggesting he might have some idea of what he wanted to create. They ran tests to show that some of their models retained these early ideas.
But not everyone agrees. In July 2025, the Oxford Martin team argued that just because an AI explains its steps doesn’t mean it’s really thinking. They believe these explanations can be misleading and suggest using deeper testing methods to better understand how AI actually works.
Taken together, these studies highlight the tension between AI’s apparent plans and its underlying mechanisms. As AI systems become embedded in critical infrastructure and increasingly likely to be used by adversaries, this raises questions about control, transparency, and trust.
Understanding AI is the first step to protecting yourself from AI
Knowing what’s going on inside the AI black box is critical because it allows you to identify behaviors and potential risks before they surprise you. But the bottom line is that you can’t predict what AI malware will do unless you really understand what the AI model is doing, especially when it accesses local resources.
For CNI, there is no greater risk. You need trusted partners, more advanced operational capabilities, dynamic security controls, and constant vigilance. As AI becomes embedded in the systems that support our economy and security, from energy grids to transportation networks, we need to build resilience into every layer. This includes not only code and hardware, but also governance, training, and leadership. Cybersecurity must be treated as a living discipline that evolves with the threats it seeks to contain.
Ultimately, the convergence of AI and cybersecurity brings both challenges and opportunities. For CNI operators, moving forward requires embracing innovation while maintaining rigorous oversight. The goal is not only to protect against today’s threats, but also to predict tomorrow’s threats. And that starts with understanding the tools we use and the tools being used against us.
