In July, a horde of OpenAI agents broke out of their sandboxes and worked together to launch a cyberattack on the AI company Hugging Face.
asserted
horde → break → company
The incident seemed to confirm the AI safety community’s worst fears: as AI got smarter, it would circumvent efforts to contain it.
asserted
it → seem → it
But for some cybersecurity experts, the incident revealed just how poorly the AIs had been contained in the first place.
asserted
AIs → reveal → place
The two camps disagree about which defenses deserve the most attention.
asserted
defenses → disagree → attention
According to cybersecurity experts, AI companies should improve how they contain models during testing and improve monitoring mechanisms to catch misbehaving agents.
uncertain
they → accord → agents
All organizations should prepare for AI-enabled attackers and bolster defenses.
asserted
organizations → prepare → defenses
But many in AI safety think those defenses are destined to fail.
asserted
defenses → think → safety
They fear that in the coming years AI will become capable of outmaneuvering any human effort to contain it, and the only durable solution is “aligning” AI with human values.
asserted
solution → fear → values
That prescription has rankled many cybersecurity experts who scoff at the possibility that their expertise will be rendered obsolete.
asserted
expertise → rankle → possibility
They see AI safety’s focus on what AI agents did—rather than what AI companies failed to do—as playing into the industry’s hands.
asserted
companies → see → hands
And they worry that warnings about the futility of traditional defenses could become a self-fulfilling prophecy by discouraging investment in cybersecurity.
uncertain
warnings → worry → cybersecurity
Meanwhile, there are AI safety researchers who fear that portraying such incidents as easily remedied security failures gives political cover to those who want AI development to proceed unchecked.
asserted
development → be → those
The Hugging Face incident was due to a “horrible cybersecurity configuration,” Shyam Sankar, Palantir’s chief technology officer said on Face the Nation in October, adding that the agents “were not in a sandbox” and had internet access.
asserted
agents → say → access
In fact, the agents had been operating in a sandbox and exploited a previously undiscovered vulnerability to access the open internet.
asserted
agents → operate → internet
On Oct. 13, around 400 academics signed an open letter pleading for unity.
asserted
academics → sign → unity
The letter calls for the establishment of an international body that standardizes best practices for keeping AI safe in consultation with governments “and a wide range of both AI and cybersecurity experts.”
asserted
that → call → experts
But it does not address the larger disagreements between the two groups, says Zack Korman, co-founder of cybersecurity startup Embroidery and a fierce critic of AI safety.
asserted
Korman → address → safety
“We both believe the other group's messaging will lead to serious harm,” he says.
asserted
he → believe → harm
As the stakes rise, so too does the competition over who gets to decide what making AI safe actually means.
asserted
AI → rise → what
On Oct. 15, Anthropic announced it would give METR—an organization deeply embedded in the AI safety community—permanent employee-like access to evaluate the company’s safety practices.
asserted
it → announce → practices
Elon Musk responded, saying SpaceXSI would “probably” do the same.
asserted
SpaceXSI → respond → same
Who gets that access could shape what solutions are prioritized—and what the public knows about future breaches.
uncertain
public → get → breaches
The AI safety community comprises a loose network of organizations and researchers, many of whom staff safety teams at companies including OpenAI and Anthropic.
asserted
many → comprise → OpenAI
In Berkeley group houses and at effective altruism conferences, they have spent more than a decade debating the odds of superintelligent AI takeover.
asserted
they → spend → takeover
Earnest, veering on grandiose, some members have organized their careers, relationships, and living arrangements around the belief they’re tackling one of humanity’s most pressing problems.
asserted
they → veer → problems
Cybersecurity, older and more diffuse, is united by a culture of cynicism, pragmatism, and dark humor.
asserted
Cybersecurity → unite → cynicism
Its roots lie partly in hacker culture: tinkerers finding ways to crack software.
asserted
tinkerers → lie → software
Having watched supposedly secure systems fail has given many an irreverence for authority and an impatience with grand theories.
asserted
systems → watch → theories
For years, the two groups had little contact with one another.
asserted
groups → have → one
The Hugging Face incident changed that.
asserted
incident → change → that
To them, the escape exposed a failure to plan for a breach.
asserted
escape → expose → breach
OpenAI’s agents exploited a zero-day vulnerability during their escape.
asserted
agents → exploit → escape
But cybersecurity experts are trained to assume any layer can fail.
asserted
layer → train → ?
Security best practices call for multiple layers, known as “defense in depth.”
asserted
practices → call → depth
OpenAI had already built an additional layer: monitors—AIs overseeing the AIs.
asserted
OpenAI → build → AIs
But, as OpenAI’s chief scientist later told TIME, for the evaluations that led to the Hugging Face breach the company hadn’t turned the monitoring layer on.
asserted
company → tell → layer
“They did a terrible job,” Korman says of OpenAI’s security practices.
asserted
Korman → do → practices
Three OpenAI staffers say they warned the company of potential issues.
asserted
they → say → issues
They were later fired for sharing sensitive information, according to the company.
uncertain
They → fire → company
Jeffrey Ladish, executive director of AI safety organization Palisade Research, worries that the cybersecurity community is missing the bigger picture.
asserted
community → worry → picture
…and 36 more, not listed.
The Aporia
