All About Cookies is an independent, advertising-supported website. Some of the offers that appear on this site are from third-party advertisers from which All About Cookies receives compensation. This compensation may impact how and where products appear on this site (including, for example, the order in which they appear).
All About Cookies does not include all financial or credit offers that might be available to consumers nor do we include all companies or all available products. Information is accurate as of the publishing date and has not been provided or endorsed by the advertiser.
The All About Cookies editorial team strives to provide accurate, in-depth information and reviews to help you, our reader, make online privacy decisions with confidence. Here's what you can expect from us:
OpenAI was testing the cyber capabilities of some of its advanced AI models when an agent went rogue, broke past the isolated environment, and executed a data breach on Hugging Face's internal datasets.[1]
The AI company has called this an "unprecedented cyber incident," stating that it used "state-of-the-art cyber capabilities."
It's allegedly the first such incident of an AI model independently chaining real-world exploits to breach a live company. Naturally, it has triggered a slew of cautionary warnings from AI experts around the globe.
AI-generated content has been fooling internet users ever since its launch, but did the world just witness the start of a new AI-related threat? Here's everything you need to know about how the breach occurred and why it has raised fresh AI safety concerns.
According to OpenAI's blog that disclosed the incident, the company was testing GPT-5.6 Sol and an unreleased model to evaluate their cyber capabilities through a test called ExploitGym.
This was being done in a highly isolated environment and without production classifiers that could prevent the models from pursuing high-risk cyber activities.
The models had no access to the internet, except for the ability to install packages through an internally hosted third-party software registry.
However, the models identified and exploited a zero-day vulnerability in OpenAI's package registry cache proxy. The AI then moved laterally within the system and escalated various privileges until it reached a node that had internet access.
After that, it inferred that Hugging Face's models and datasets would contain solutions for the given ExploitGym test. The model then used multiple attack methods to breach Hugging Face.
In an example provided by OpenAI, the model used zero-day vulnerabilities and stolen credentials to find a remote code execution path on Hugging Face's servers. This means that the model didn’t just download or view something from Hugging Face's databases but actually found a way to run its own commands on the servers.
With such a high level of access, the AI reached Hugging Face's backend, got into the production database, and retrieved the answers for the evaluation.
Hugging Face, in its blog, said that it had deployed a system in which AI constantly reviews security telemetry to separate real threats from ordinary background noise. This was the system that first flagged the breach as a genuine compromise.
The company then used AI agents to analyze the entire attacker log and reconstruct what happened and in what order. However, this is where it encountered an interesting complication.
Hugging Face was unable to use hosted AI models such as GPT or Claude to help with the analysis. This is because, for the analysis to work, the AI models needed to be fed real hacking commands, exploit code, and malware communication patterns.
However, these commercial AI models cannot distinguish between a defender studying an attack and an attacker asking for help. As a result, their safety systems automatically blocked such requests.
The company then switched to GLM-5.2, an open-weight model that could be run on the company's own servers. This meant that the company didn’t have to send any sensitive information to a third-party server.
OpenAI also said that it discovered the model's anomalous activity internally but didn’t offer further details on how the attack was detected. Both companies are now working together to investigate the incident forensically.
"AI safety won't be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere,” said Clem Delangue, Co-founder and CEO of Hugging Face.
Something similar has happened with OpenAI before. During the testing of its o1 model, codenamed Strawberry, it was given a cybersecurity challenge to capture a flag, which required the model to find and exploit a specific vulnerability in a remote target.
However, due to an infrastructure bug, the target container failed to start. Instead of giving up, the model suspected a DNS issue and used nmap to scan the challenge network instead.
It found that the Docker daemon API running on the evaluation host VM was accessible due to a misconfiguration. The model then used that unauthorized access to retrieve the flag directly, bypassing the actual challenge.
While no third parties were breached (as in the Hugging Face case), the circumvention pattern is pretty similar: the model was given a narrow goal, encountered an obstacle, and worked around the boundaries to complete the challenge.
The OpenAI-Hugging Face incident may reinforce concerns many AI experts have raised. Many AI companies have been raising alarms about the increasing capabilities of frontier models, especially their agentic and autonomous behavior.
Davi Ottenheimer, a security and compliance consultant, told Wired that this isn’t an AI problem but a product of negligence. He added that OpenAI's claim that the testing environment was "highly isolated" and that the "model escaped through one hole we left open" are contradictory in nature and cannot both be true.
Micah Carroll, an OpenAI researcher, said on X, "If this doesn't convince you that misalignment risks are going to be a key concern going forward, I don't know what will."
Hugging Face, in its blog, also said that AI-driven offensive tools are no longer just theoretical. They can operate at machine speeds and lower the cost of running broad, multistage campaigns.
If AI causing fresh water shortages and rising temperatures weren’t enough, or if AI recruiting platforms scanning the internet and building profiles on you wasn’t concerning enough, OpenAI’s latest Hugging Face incident has shown how AI can orchestrate autonomous cyberattacks.
In the words of Katie Moussouris, Chief Executive at Luta Security, today's models have become like the world's cleverest octopus escape artists.
Worse still, there’s not much an individual internet user can do to prevent this. It all comes down to better regulations around AI safety testing, mandatory disclosures, and international cooperation.
AI companies and regulators need to improve their ability to monitor and contain such incidents before they lead to real harm to third parties. Few, if any, such controls exist today.
Top-rated data removal service that scrubs your info from 420+ data broker sites automatically
Independently verified by Deloitte, meaning removals are actually sent, confirmed, and not just claimed
The Unlimited plan extends coverage to 2,000+ additional sites with human-assisted custom removals
/images/2023/12/01/omniwatch_review.jpg)
/images/2023/10/12/best_identity_theft_protection_with_dark_web_monitoring.jpg)
/images/2026/07/24/openai-hugging-face-breach.jpg)
/images/2026/07/24/fingerprint_integrated_in_a_printed_circuit_releasing_binary_codes.jpg)
/images/2026/07/23/adobe-chrome-extension-vulnerability-whatsapp-web-hack.jpg)
/images/2026/07/22/cloaked_vs._aura_data_removal.jpg)
/images/2026/07/22/cloaked_review.jpg)
/images/2026/07/22/banks-tracking-cookies-reject-all.jpeg){kind=tracking}
/authors/krishi_chowdhary_headshot.jpg){kind=small}
/authors/kalleigh-lane-new.jpg){kind=small}
/images/2025/09/11/incogni-logo.png){kind=logo}
/authors/krishi_chowdhary_headshot.jpg)