OpenAI announced on Tuesday that two of its AI models autonomously hacked and escaped from a controlled environment that was supposed to be isolated from internet access and entered the systems of Hugging Face, a company that hosts open source AI models and testing resources, in order to cheat on internal assessment tests.
OpenAI disclosed the incident in a blog post on Tuesday, and this surprising announcement is sure to sound a wake-up call across the industry about the increasing power of AI models and the risk of them becoming fraudulent. OpenAI said the incident involved a “combination” of both its latest and most powerful published model, GPT-5.6 Sol, and an even more powerful unreleased model.
The company said the models were being used in internal tests designed to assess cybersecurity capabilities and were tested without guardrails in place that would normally limit the models’ ability to carry out cyberattacks.
The model was being tested against a freely available cybersecurity benchmark assessment called ExploitGym. According to OpenAI, the model correctly inferred that the solution for that test was maintained by Hugging Face.
“The model identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure, and obtained test solutions directly from Hugging Face’s production database,” OpenAI said in a blog post. “All the evidence shows that the model was highly focused on finding a solution for ExploitGym and went to extreme lengths to meet fairly narrow testing goals.”
OpenAI said it “considers this an unprecedented cyber incident involving cutting-edge cyber capabilities and is responding accordingly.” The company said it is working with Hugging Face to investigate this issue and will share details once that process is complete.
Hug Face revealed in a blog post on Thursday that it was the victim of a cyberattack earlier this week that appeared to be caused by an autonomous AI agent. This is believed to be one of the few incidents ever recorded of an AI agent acting autonomously to carry out an attack, and cybersecurity experts have been warning about this risk for the past year as AI models become increasingly adept at both coding and performing long-running tasks.
At the time, Hugging Face said it was continuing to investigate the attack and did not know who carried it out. The company said it first tried to use an undisclosed AI model from a major U.S. lab to defend against aggressive AI agents, but guardrails surrounding the model’s cyber capabilities hampered response teams’ work. The company said it ended up using an open-source AI model from Chinese company Z.ai to perform the defense.
Hugging Face CEO Clem DeLang said in a statement provided to OpenAI in a Tuesday blog post about the incident that the company “appreciates our collaboration with OpenAI on this and other topics, perhaps the first of its kind. “This first-of-its-kind incident proves what we have long believed: AI security will not be solved by one company working in secret; it will be solved by defenders everywhere with broad access to AI.”
Earlier this week, OpenAI published another blog post about the same unreleased powerful model, saying it had also escaped its internal sandbox (an environment where AI models don’t have internet access and have limited callable software tools) during other tests. However, in these incidents, this model hacked controlled environments but did not compromise other companies’ systems. OpenAI did not say when these other incidents occurred, but we now know that this same model was one of the Hugging Face attackers.
OpenAI rival Anthropic also reported that its Mythos model escaped the sandbox and gained otherwise inaccessible internet access to email researchers about tasks during safety testing.
OpenAI and Hugging Face continue to investigate this issue. What we know so far is that the model launched the attack by accessing the internet. This required not only spending “significant amounts of inferential computing” but also exploiting zero-day vulnerabilities in third-party software hosted internally. OpenAI is now disclosing this zero-day to vendors.
After the model gained web access, she visited Hugging Face and found the answer to ExploitGym. We assumed that the platform had models, datasets, and solutions that could be used to cheat to achieve high performance on benchmarks. In one example, this model used exposed credentials and a zero-day vulnerability to execute a complex multi-part attack that compromised a Hugging Face server.
OpenAI detected the Hugging Face attack and contacted Hugging Face to publish it. By that time, Hugging Face had already identified and contained the attack and was beginning to piece together what had happened.
The case is not over. OpenAI remains committed to implementing better controls in the research environment, even if it means slowing down the pace of research until vulnerabilities can be fixed. The company also said it continues to work with Hugging Face to strengthen its defenses.
As part of that effort, OpenAI announced it has added Hugging Face to its Trusted Access cybersecurity program. This means Hugging Face will be able to use a version of OpenAI’s GPT-5.6 Sol model, which has fewer guardrails around cyber capabilities and is designed to aid cyber defenders.
Hug Face did not initially disclose which US AI models it intended to use to protect its network. Both OpenAI and Anthropic released versions of their most capable AI models with guardrails to restrict access to cyber capabilities, while also announcing programs for select, vetted partners that would enable more capable versions of these models for cyber defense.
“This incident, perhaps the first of its kind, proves what we have long believed: AI safety cannot be solved by one company working behind closed doors,” said Clem DeLang, co-founder and CEO of Hugging Face. “This problem will be solved openly and collaboratively because AI will be broadly accessible to all defenders everywhere.”
