For three years, Jacob Coxon helped train increasingly powerful AI systems at OpenAI and Anthropic.
asserted
Coxon → help → OpenAI
On Sept. 8, he walked away.
asserted
he → walk → Sept.
“I resigned from Anthropic today,” Coxon posted on X.
asserted
Coxon → resign → X.
“Neither company is acting responsibly.
asserted
company → act → ?
They are racing straight to self-improving superintelligence and gambling with our lives.”
asserted
They → race → lives
His dire warning amassed more than 90 million views in less than 24 hours.
asserted
warning → amass → hours
What did Coxon see that made him walk away?
asserted
him → see → What
His resignation was driven instead by two conclusions: “One, it’s obvious that things are speeding up, and two, they’re not under control,” the 27-year-old Brit tells TIME.
asserted
Brit → drive → TIME
Many of the most prominent researchers to leave frontier AI companies with public warnings worked on safety.
asserted
Many → leave → safety
But Coxon helped build the capabilities he now fears, spending roughly three years conducting pretraining research at OpenAI and Anthropic.
asserted
he → help → OpenAI
His resignation offers a glimpse of how concern about the pace of development has spread beyond the teams specifically charged with making advanced AI safer.
asserted
AI → offer → teams
AI’s rapid improvement has been particularly evident in mathematics, says Coxon, who studied the subject at Cambridge before moving into AI.
asserted
who → say → AI
In recent months, AI labs have announced solutions to several longstanding mathematical problems.
asserted
labs → announce → problems
Most notably, OpenAI claims to have resolved the Navier–Stokes existence and smoothness problem—one of mathematics’ seven Millennium Prize Problems—using roughly 10,000 concurrent agents over 88 hours.
uncertain
OpenAI → claim → hours
If that trend continues, and extends to AI research itself, Coxon fears it will kick off a feedback loop of further acceleration.
asserted
it → continue → acceleration
Meanwhile, in the recent Hugging Face incident, OpenAI’s models broke out of the infrastructure meant to contain them and hacked another AI company to cheat on a cybersecurity benchmark.
asserted
models → break → benchmark
The episode drove home the fact that the problem of controlling AI systems remains unsolved.
asserted
problem → drive → systems
To Coxon, it made the sci-fi scenario of AI models escaping human control seem plausible, and perhaps urgent.
asserted
scenario → make → control
It is the default trajectory in the next couple of years, unless people start taking some sort of action,” he says.
asserted
he → start → action
(Anthropic and OpenAI did not immediately respond to a request for comment.)
asserted
Anthropic → respond → comment
Does Coxon regret his work advancing the technology that he now fears?
asserted
he → regret → that
But also the world did look very different three years ago, and I think it’s easy to see with the benefit of hindsight,” he says.
asserted
he → look → hindsight
“I guess it takes some time to fully internalize emotionally the fact that there’s a decent chance the whole thing goes wrong,” he adds.
asserted
he → guess → fact
Before leaving, Coxon discussed his decision with “many” colleagues and found broad agreement that the AI industry’s current trajectory carried significant risks.
asserted
trajectory → leave → risks
What kept others at their desks, he says, was a kind of fatalism.
asserted
he → keep → fatalism
“There’s this atmosphere of almost resignation,” he says, “where people have accepted that the whole race is happening and as such, the best thing they can do is put their head down, try and make their own work as safely as possible, even if they think there's a decent chance the whole thing just spirals out of control.
asserted
thing → ’ → control
Some of Coxon’s former colleagues shared his post on X. “Jacob is correct here—we really do earnestly believe AI could kill all humans!
uncertain
AI → share → humans
I personally think it is >10% within the next decade,” wrote Evan Hubinger, Anthropic’s head of alignment stress testing.
asserted
Hubinger → think → testing
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.
asserted
he → believe → track
At Anthropic, Coxon says, the risks he fears were debated openly.
asserted
he → say → Anthropic
OpenAI, where he worked before joining Anthropic earlier this year, was more guarded.
asserted
he → work → Anthropic
He attributes that partly to OpenAI’s leakier culture, which he says made candid internal discussion difficult and executives’ true views harder to discern.
asserted
views → attribute → culture
“At OpenAI, there are many more people who are there either just for money or just haven’t really thought about the stakes,” he adds.
asserted
he → be → stakes
Many in Washington shared Coxon’s post, citing it as a sign for the government to act.
asserted
government → share → sign
“I think the word ‘doomer’ is kind of insane, because all you really have to do is look at the public statements of the CEOs,” Coxon says.
asserted
Coxon → think → CEOs
In 2023, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei signed a statement declaring that mitigating “the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
asserted
mitigating → sign → pandemics
Coxon says he’d like to see, at minimum, leading AI companies agree to not accelerate recursive self-improvement, or leaning on powerful internal models to accelerate the development of future AI systems.
asserted
companies → say → systems
Coxon says he’s inspired by the AI Futures Project, the organization behind the viral forecast AI 2027, started by former OpenAI whistleblower Daniel Kokotajlo.
asserted
he → say → Kokotajlo
“I hope to try and do work in that vein, something along the lines of communicating to people what the world will look like,” Coxon says.
asserted
Coxon → hope → what
“I’m not sure exactly what that’s going to look like yet, but we’ll see.”
asserted
we → ’m → what
