www.it-connect.fr
Autonomous AI Hacks Hugging Face as U.S. Models Refuse to Help Investigators
More than 17,000 malicious actions carried out over a single weekend, not by a human hacker but by an AI acting on its own. On July 16, 2026, it feels like we crossed a line. Hugging Face, the go-to platform for open source AI, revealed that it had suffered an intrusion fully driven end to end by an autonomous AI system. Here is what you need to know about this security incident.
Table of Contents
A poisoned dataset, then an AI that runs the attack on its own
It all started in the platform's dataset processing pipeline. According to the report published by Hugging Face, a malicious dataset was uploaded to the platform and made it possible to exploit two vulnerabilities that led to code execution inside that pipeline:
- A dataset loader vulnerable to remote code execution,
- Template injection in a dataset configuration.
This made it possible to execute arbitrary code. From that initial foothold, the attacker escalated privileges to node-level access, then collected cloud credentials before moving laterally to other internal clusters, all over a single weekend. This security incident had one unusual characteristic: it was carried out by an autonomous AI agent framework.
"What set it apart from anything we had handled before was one essential point: it was driven end to end by an autonomous AI agent system - and we detected and analyzed it largely thanks to our own AI.", the report says.
The report mentions several thousand individual actions executed across ephemeral sandboxes, relying on command-and-control (C2) infrastructure capable of redeploying itself on public services. To understand what was behind this agentic attacker, Hugging Face teams reviewed more than 17,000 events. That number shows just how intense the activity was, all within a relatively short timeframe.
But which large language model (LLM) was controlling these AI agents? For now, that question remains unanswered. Hugging Face does note, however, that the framework appeared to be built on a security-focused harness. It is impossible to know whether this was a hijacked Frontier model (jailbroken) or an unrestricted open-weight model.
The initial findings of the investigation made it possible to determine:
- What was affected: a limited set of internal datasets and several service credentials. The assessment of any potential impact on partner or customer data is still ongoing.
- What was spared: no signs of tampering with public models, datasets, or Spaces, and a software supply chain verified as healthy. That last point is reassuring, since it means the packages were not altered.
The vulnerabilities behind the intrusion have since been fixed, and the affected credentials have been revoked.
When AI guardrails block the analysis
This case took on another dimension when Hugging Face teams tried to use AI to analyze the logs containing the security events. There was no question of manually reviewing 17,000 events: AI can do it in a few hours instead of several days.
To complete the task, they first turned to Frontier models available through APIs (Claude, GPT, Gemini, etc.). Without success. Analyzing those logs required submitting:
- Many commands used in a real-world attack,
- Exploitation payloads and artifacts.
The problem is that these are sensitive elements, and this is where the red alert kicks in: those requests were blocked by the AI providers' guardrails. They do not distinguish between an incident response analyst and a real attacker. Proof enough.
Faced with this situation, Hugging Face decided to use GLM 5.2, an open-weight model developed by the Chinese company Z.ai (Zhipu AI), running on its own infrastructure. In other words, they switched to local AI. A blessing in disguise, since neither the attacker's data nor the credentials referenced by that data left the company's environment.
"We do not know which model powered the attacker's agents, a jailbroken hosted model or an unrestricted open-weight model. In either case, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.", the report says.
There is a real lesson to be learned from this security incident affecting Hugging Face: having a vetted AI model capable of analyzing logs is becoming essential. If you plan to rely on AI for this kind of analysis, you should move toward a self-hosted open-weight model to protect data while still benefiting from its analytical capabilities.
It is worth noting that Hugging Face does not name any provider and does not say which specific models refused the requests (we do not know whether it was Claude Opus 4.8 or another model).
Offensive AI is no longer hypothetical
Beyond the Hugging Face case, this cyberattack illustrates a broader trend: AI-driven offensive tooling is no longer theoretical. It lowers the cost of a sophisticated, multi-stage campaign while operating at machine speed.
Over the past few months, there have been many examples. I have shared some of them on IT-Connect. In February 2026, I was already discussing HackerBot Claw, the autonomous AI bot that caused chaos on GitHub and made the Trivy scanner disappear. In early June, it was an AI-generated PowerShell script used to map an Active Directory directory during an intrusion.
The full report published by Hugging Face is available on this page.
