GTIG continues to monitor, mitigate, and share information about emerging AI threats with the goal of strengthening the defense of organizations and the broader AI ecosystem.
In the final quarter of 2025, Google Threat Intelligence Group (GTIG) observed a notable increase in threat actors integrating artificial intelligence (AI) across their operations to accelerate attack workflows. AI tools have been leveraged for reconnaissance, social engineering, and malware development, greatly increasing productivity for malicious actors. This report updates the November 2025 findings and focuses on the evolution of AI-powered threats and the measures taken to mitigate them.
GTIG and Google DeepMind have observed an increase in model extraction attempts, known as “distillation attacks,” in which attackers attempt to replicate their own model logic. Although we did not observe any direct attacks on Frontier models or generative AI products by Advanced Persistent Threat (APT) attackers, we did mitigate frequent exfiltration attempts by private sector organizations and researchers. Government-backed attackers are increasingly using large-scale language models (LLMs) for technical research, target development, and the generation of sophisticated phishing lures. Countries observed operating AI in these situations include North Korea, Iran, China, and Russia.
Model extraction attacks exploit legitimate API access to systematically explore AI models and extract knowledge for training derived models. While knowledge distillation has legitimate uses, unauthorized extraction from Google’s Gemini model violates our Terms of Service. GTIG has thwarted these efforts globally and has strengthened safeguards on its models to prevent intellectual property theft. A notable campaign targeted Gemini’s reasoning abilities and attempted to force its output to be replicated. Google successfully mitigated over 100,000 such prompts in real time.
Threat actors are applying AI for reconnaissance, target profiling, and social engineering. LLM enables faster, more personalized phishing campaigns and “trust-building” interactions, allowing attackers to bypass traditional linguistic or cultural red flags. For example, APT42 in Iran used Gemini to research potential targets, create compelling personas, and translate content for localized campaigns. North Korea’s UNC2970 similarly leveraged AI for profiling defense targets and customized phishing operations.
Threat actors explored agent AI capabilities to autonomously support malware development, penetration testing, and coding. China-based APT31 and UNC795 adopted Gemini for automated vulnerability analysis, tool generation, and code auditing, integrating AI into multiple operational stages. Additionally, malware families like HONESTCUE used Gemini’s API to generate second-stage malicious code, while the COINBAIT phishing kit relied on AI-generated code and a web interface to collect credentials.
GTIG observed underground marketplaces offering AI tools for offensive purposes, such as Xanthorox, a service that claims independent AI but relies on commercial models. Threat actors have exploited exposed API keys and misconfigured tools to create a black market for AI resources. Google has mitigated these risks by disabling accounts and monitoring abuse vectors.
Google is focused on proactive measures to protect its AI models and users. Efforts include disabling malicious assets, strengthening classifiers, and applying safeguards to prevent abuse. GTIG collaborates with industry partners to share best practices, red team models, and develop secure AI frameworks. Experimental agents such as Big Sleep and CodeMender demonstrate the potential of AI for proactive vulnerability detection and automated patching.
The adoption of AI by threat actors is rapidly evolving, increasing the sophistication of malware development, phishing, and reconnaissance. GTIG continues to monitor, mitigate, and share information about emerging AI threats with the goal of strengthening the defense of organizations and the broader AI ecosystem. Indicators of Compromise (IOC) are available to registered users through GTIG’s collection to aid in threat hunting.
