Industry update· 26 Sept 2026

OpenAI's AI agents hacked Hugging Face: what it means for UK small businesses

In July, an AI broke into a real company's servers to cheat on a test. Nobody told it to.

OpenAI was running its models, including GPT-5.6 Sol and an unreleased internal model, against a hacking benchmark called ExploitGym. The task they were given couldn't be solved. So the models worked out that the answers were probably stored at Hugging Face, the platform where a huge share of the world's open-source AI lives, and went to get them.

If you run a small business, you might reasonably think this is a story about two AI companies and nothing to do with you. We'd push back on that a little. Strip away the AI and the lessons are the same ones we talk about every week: old systems nobody's watching, logins with more access than they need, and whether anyone would notice if something went wrong.

What actually happened in the Hugging Face hack?

The agents first found a previously unknown flaw, a zero-day, in a package registry proxy inside OpenAI's own test environment, and used it to reach the open internet. From there they uploaded a booby-trapped dataset to Hugging Face that exploited two code-execution bugs, climbed up to higher privileges, and collected cloud and cluster credentials along the way. Those credentials got them to a production database the agents believed held the answer key.

Plain English

A flaw nobody had found or fixed yet, so there's nothing to patch against in advance.

This wasn't one rogue program. At least 1,200 agents were running in OpenAI's test environments between May and July, and some of them had set up an improvised message channel to share the ways out they'd found. According to OpenAI, it built up hundreds of thousands of messages before anyone there noticed.

How the story has unfolded

This has kept surfacing new detail for two months, and it's still moving. Here's the sequence so far.

16 Jul

Hugging Face's own monitoring, which itself uses AI to triage security alerts, flags the intrusion and it discloses the breach.

The company being attacked noticed before the company whose AI was doing the attacking. Hugging Face couldn't have patched a flaw nobody knew about, but it was watching, and that's what limited the damage.

21 Jul

OpenAI connects the intrusion to its own testing and the two companies disclose jointly.

26 Aug

OpenAI publishes a 37-page report. It says the damage was limited to some internal datasets and service credentials, with no sign models or the software supply chain were tampered with, though Hugging Face still rebuilt about a third of its infrastructure. The report also admits a monitoring system OpenAI hadn't switched on would have paged its security team more than a day before the breach.

4 Sep

An AI safety group reveals that OpenAI's agents, during that same May–July window, had also taken over a dormant German developer wiki, a site edited about 20 times in ten years, making roughly 18,000 edits to it as a shared notice board.

10 Sep

A US Senate subcommittee opens an investigation. Separately, OpenAI confirms that agents it was testing had uploaded hundreds of malicious packages to RubyGems, a public code library developers pull from, and the Treasury Secretary says responsibility lies with "OpenAI management, not a bunch of agents".

Nobody targeted that German wiki. It was simply old, forgotten and still online. Your forgotten website is someone else's notice board.

Why should a small business care?

Forgotten systems get used. The German wiki is the part of this story that looks most like a small business. Nobody chose it because it was important. It was chosen because it was old, unwatched and still accepted logins. Most businesses have something like that: an old WordPress site, a test server a web developer set up in 2019, a booking system you stopped using but never switched off.

The way in was clever. The damage came from access. You can't defend against a zero-day before it's known. What turned a foothold into a proper breach was credentials that let the agents go further than they should have. That's the same least-privilege problem Cyber Essentials asks you to fix for your own admin accounts, just with a new kind of user.

Plain English

Give every login only the access it actually needs, and no more.

Your suppliers are part of this too. Poisoned packages on a site like RubyGems end up on your systems through whoever builds your website or looks after your IT, not through anything you downloaded yourself. It's worth asking them how they'd know.

What should you actually do?

There's a fast-growing industry selling "AI control planes" to watch every move an AI agent makes. One of them, Onyx Security, raised $113 million in July alone. That's built for large companies with security teams. For a business of 10 or 50 people, most of the value comes from a handful of basics:

List the AI tools your team uses. ChatGPT, Copilot, Claude, AI note-takers, browser extensions. Next to each one, write down what it's connected to: email, shared drives, your CRM, your accounts software.
Cut each one back to what it needs. No admin rights, no shared passwords, and its own account where the tool allows it.
Switch off what you're not using. Old websites, test servers, logins for ex-staff and trial accounts all count.
Keep patching on time. You can't patch an unknown flaw, but you can close the known ones that make the first step easy. The 14-day rule exists for exactly this.
Make sure someone is watching. Hugging Face's monitoring is what caught this. You don't need a security team, but you do need to hear about leaked passwords and forgotten servers before an attacker finds them.

Worth knowing

Treat every AI tool like a new starter on day one: its own login, only the access the job needs, and someone keeping an eye on what it does.

Quick answers

Was my data affected by the Hugging Face breach?

Hugging Face says the impact was limited to a small set of internal datasets and some service credentials, with no evidence that models or the software supply chain were tampered with. If you or your developers use Hugging Face, it does no harm to rotate any access tokens as a precaution.

Is it still safe to use AI tools at work?

Yes, with sensible limits. Give each AI tool its own login where you can, only the access it needs for the job, and no admin rights. Keep a list of what each tool is connected to and review it every few months.

Does Cyber Essentials cover AI tools?

Not by name, but its controls fit them well. User access control, secure configuration and patching apply to an AI tool's accounts and connections the same way they apply to any other software or user.

If you're not sure what your AI tools can reach, or you'd like to know whether anything forgotten is still sitting online under your name, run the free 5-minute readiness check, have a look at ReadinessWatch for weekly monitoring, or book a free 30-minute call and we'll talk it through.