Hugging Face, the world’s largest open source artificial intelligence (AI) repository, has revealed that it has been subjected to a sophisticated hacking attack, caused entirely by an “AI agent.” Hugging Face also revealed that it used a Chinese open weight AI model to mitigate the attack after the US-made model blocked requests to neutralize the attack due to guardrails.
Hugface explains how AI attacks unfold
According to Hugging Face, the breach began with a vulnerability specific to the AI hosting infrastructure, or data processing pipeline. The malicious dataset injected into the system exploited two separate code execution flaws. Once inside, the autonomous AI agent escalated access to take control of internal server nodes, collected security credentials, and moved laterally between multiple internal server clusters over the weekend.What attracted particular attention was the scale and speed of the hacking incident. According to Huggin Face, the hackers deployed an autonomous agent framework to perform thousands of coordinated actions across a temporary sandbox.Second, AI systems dynamically migrated command and control operations across public cloud services, bypassing traditional security filters. Hugging Face confirmed that while limited internal datasets and service credentials were compromised, there is no evidence that any publicly available user-facing models, datasets, or software packages have been tampered with.“The intrusion began in the AI platform’s proprietary exposed data processing pipeline. “A malicious dataset exploited two code execution paths in our dataset processing (remote code dataset loader and template injection into dataset configuration) to execute code on processing workers,” the company said.
‘AI safety guardrail blocked the defender
As Hugging Face engineers rushed to analyze server logs, they encountered a roadblock. The team initially tried to analyze malicious code using a commercial top-of-the-line AI model via a cloud API. However, the safety guardrails built into these commercial AI services flagged the actual attack payload as harmful content, immediately locking out cybersecurity responders. To avoid lockouts, Hugging Face ran GLM 5.2, an openweight AI model by Chinese technology company Z.ai, locally on its private infrastructure. This allowed defenders to freely analyze malicious attack data while ensuring that sensitive internal tokens never left the secure environment.“While the attackers were bound by a no-use policy, our own forensic work was blocked by the guardrails of the first hosted model we tried,” Hugging Face noted, pointing to a significant gap in corporate cybersecurity.
