My fiancé works at Anthropic.
asserted
fiancé → work → Anthropic
Last week, we learned that a group of OpenAI models broke out of their test environment and hacked into Hugging Face to steal the answers to a benchmark they were being tested on.
asserted
they → learn → benchmark
It’s the first publicly known case of an autonomous AI agent system designing and successfully executing an attack like this, and the fallout is stretching into this week.
asserted
fallout → ’ → week
One, AI safety experts noted that the incident signaled that OpenAI’s models now carry a “critical” capability threshold for cybersecurity, according to the company’s own preparedness framework.
uncertain
models → note → framework
(The framework, which OpenAI updated in April 2025, represents an effort at self-regulation in a world where AI companies can still largely build whatever they want.)
asserted
they → update → whatever
The document states that a model will represent a critical risk when “A tool-augmented model can identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention.”
asserted
model → state → intervention
This seems to be what happened with the Hugging Face attack; OpenAI has said its models identified and exploited a zero-day vulnerability as part of the attack.
asserted
models → seem → attack
This matters because the policy states that should OpenAI develop a model with critical capabilities, it will “halt further development” until “we have specified safeguards and security controls standards that would meet a Critical standard.”
asserted
that → matter → standard
So does this one qualify?
asserted
one → qualify → ?
The company didn’t respond when I asked today, though it told Fortune that it is conducting a “thorough review” and later plans to “publish a technical report of our learnings for everyone.”
asserted
it → respond → everyone
Two, the incident has produced an industry alliance.
asserted
incident → produce → alliance
On Monday, Nvidia launched the Open Secure AI Alliance, a group of more than 40 companies and other organizations that are pledging “to develop and share open technologies, techniques and tools to safeguard software and agents in the age of AI.”
asserted
that → launch → AI
The group came about over frustrations that Hugging Face was unable to use frontier models from OpenAI or Anthropic to defend against the attackers, and had to use Chinese models instead.
asserted
Face → come → models
(The Trump administration forced the companies to limit US models’ cybersecurity capabilities as a condition of releasing them.)
asserted
administration → force → them
And while the alliance should mostly be seen as a lobbying effort — a way to position open-source models as safety tools amid regulatory pressure to place limits on them — it illustrates how the incident has galvanized a broad response from the tech industry.
asserted
incident → see → industry
Three, we continue to learn new details about misalignment problems with OpenAI’s models.
asserted
we → continue → models
In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter.
uncertain
agent → leave → matter
The notes, found in a part of OpenAI's infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said.
uncertain
people → find → constraints
Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.
asserted
one → yield → people
Reuters could not establish if these incidents were linked to the rogue agent that began escaping on July 9 and attacked Hugging Face on July 11.
II.
uncertain
that → establish → July
On one hand, this is hardly the first worrisome behavior we have seen from AI models.
asserted
we → see → models
In 2024 researchers found that when trained to do something it didn't want to do, Anthropic's Claude would "strategically pretend to comply with the training objective to prevent the training process from modifying its preferences."
asserted
Claude → find → preferences
Last year, the system card for Claude Opus 4 revealed that when the model was led to believe it would be retrained by a hostile actor, it tried to steal and back up its own model weights.
asserted
it → reveal → weights
But those examples were caught during controlled testing.
asserted
examples → catch → testing
The Hugging Face attack demonstrated the degree to which efforts to align models are not keeping pace with their development.
asserted
efforts → demonstrate → development
A model that can escape its sandbox could eventually exfiltrate its weights, for example, and set itself up somewhere else on the internet.
uncertain
that → escape → internet
And so the idea that these models are writing notes to each other to help with future breakout efforts feels like a red-alert moment for AI regulation.
asserted
models → write → regulation
But it was not universally received as such.
asserted
it → receive → ?
When I posted about the note-leaving on Bluesky, I was taken aback by the amount and variety of vitriol I received in response.
asserted
I → post → response
Bluesky's hostility to non-consensus views is by this point well known.
asserted
hostility → know → point
But the degree to which many educated people seem to dismiss AI safety concerns almost entirely despite the models' rapidly advancing capabilities seems worrisome.
asserted
people → educate → capabilities
The arguments, such as they are, fall into a few camps.
asserted
they → fall → camps
"This is basically a marketing pitch for their models," a user named Coffee Indiana told me.
asserted
user → name → me
"Private company that depends on investment to continue operations says it has super duper top secret hyper powerful model.
asserted
it → depend → model
Two people familiar with the operation confirm how awesome it is."
This is ridiculous.
asserted
This → confirm → operation
OpenAI lost control of its models, they hacked one of the company's partners, and the company didn't notice for several days.
asserted
company → lose → days
Law enforcement got involved.
asserted
enforcement → involve → ?
"Follow the money" can feel like a smart thing to say, but it can just as often serve as a gateway to delusional conspiracy theories.
asserted
it → follow → theories
Climate deniers often suggest that scientists are "in it for the money," for example.
asserted
scientists → suggest → example
In truth, they are simply observing reality.
asserted
they → observe → reality
…and 68 more, not listed.
The Aporia
