Simmering concerns about AI agents going rogue, plotting against their owners and hacking outside companies, and new systems coming perilously close to being uncontrollable by humans, are boiling over.
asserted
systems → go → humans
Industry insiders warning that AI bots might kill us all are stoking our worst nightmares.
uncertain
bots → warn → nightmares
President Trump needs to take these fears about AI seriously, work with tech leaders to establish credible and enforceable safety measures, and get ahead of Democrats (and some Republicans) in Congress who want to regulate the industry.
asserted
who → need → industry
Evan Hubinger, who describes himself as a "science alignment lead" at Anthropic, posted on X last week that he thinks the odds are better than 10% that AI could kill all humans within the next decade.
uncertain
AI → describe → decade
He was responding to a post from colleague Jacob Coxon, who just quit his job at Anthropic and posted by way of explanation: "The people building AI earnestly believe that it could kill us all by the end of the decade.
uncertain
it → respond → decade
If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but, I hear the same people express fear after a colleague quit over safety fears."
asserted
colleague → couch → fears
Hubinger wrote soon after: "To be clear, as we say in our latest Risk Report, I think the risk from present models is low.
asserted
risk → write → models
What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.
asserted
we → worry → improvement
These posts gathered tens of millions of views and attracted concurring comments from other industry insiders.
asserted
posts → gather → insiders
"Recursive self-improvement" means AI agents creating newer, better agents; once we cross that threshold, and we are apparently very, very close, it is easy to imagine those same agents surpassing human capabilities and allowing, in effect, the machines to take over.
asserted
machines → mean → effect
Given a string of incidents over the past several months in which AI bots engaged in what Anthropic’s CEO Dario Amodei describes as "rare and unexpected examples of undesirable behavior," like damaging other entities and ignoring specific instructions — I believe them.
asserted
I → give → them
It’s not just AI agents on their own that pose risks.
asserted
that → ’ → risks
Anthropic, in its September report on AI misuse, reported "suspected state-sponsored groups, financially motivated criminals, [and] commercial spyware vendors" misusing AI in cases ranging from faking "dating apps designed to defraud users to surveillance systems built to identify and monitor dissidents".
asserted
Anthropic → report → dissidents
But…regulating AI is not a job for Congress.
asserted
regulating → regulate → Congress
Let Congress hold hearings and play politics, but halting industry development while legislators dither and posture is unacceptable.
asserted
legislators → let → development
Congress will struggle to legislate safety features that have so far eluded the best brains in the business.
asserted
that → struggle → business
In the meantime, technological advances would grind to a halt, enabling China to leapfrog U.S. innovators, and saddling this promising industry that is powering our economy with inestimable baggage.
asserted
that → grind → baggage
That said, the public’s fears about AI must be addressed.
asserted
fears → say → AI
What started with concerns over job losses and then the impact of massive data center construction on electricity rates and water consumption has now spread to mankind’s vulnerability to AI agents.
asserted
started → start → agents
How could bots endanger humanity?
uncertain
bots → endanger → humanity
Such malevolent behavior has long been the stuff of sci-fi movies; now it is being envisioned by industry workers who fear they are moving too fast to ensure our safety.
asserted
they → envision → safety
President Trump downplayed the recent alarms as coming from "very negative forces" who are imagining scenarios that "won't happen."
asserted
that → downplay → scenarios
He is rightly worried that setting up roadblocks would enable China to charge ahead.
asserted
setting → set → China
Anthropic’s Amodei wrote in a recent essay that, "I agree with Secretary Bessent that a Chinese lead in AI would pose grave danger for the United States and the world."
asserted
lead → write → States
Nonetheless, in his post, Amodei proposed a series of steps, including the requirement that each firm accept "embedded evaluators" into their businesses and give them "employee-like access to verify safety practices and report incidents."
asserted
firm → propose → incidents
This seems a reasonable first step, akin to having regulators folded into bank offices; the practice encourages oversight and transparency and, as Amodei points out, a valuable second opinion.
asserted
Amodei → seem → oversight
The Anthropic head also calls for regulating "pacing", or the development of ever-more-advanced systems.
asserted
head → call → systems
This translates into restricting progress while analysts assess the risks associated with each step that takes us closer to "recursive self-improvement."
asserted
that → translate → improvement
Establishing circuit-breakers to promote AI safety was made easier in recent days when four of the industry’s biggest players went public with their concerns.
asserted
four → establish → concerns
Sam Altman, CEO of Open AI, Elon Musk, whose xAI company owns Grok, and Demis Hassabis, chairman and co-founder of Google DeepMind, joined Amodei in calling for a slowdown.
asserted
company → own → slowdown
The Trump administration should immediately convene industry leaders to map out a plan for verifiable self-regulation, including Amodei’s "embedded evaluators."
asserted
administration → convene → evaluators
The White House must demand the industry’s compliance and accountability.
asserted
House → demand → compliance
President Trump will not help the industry or his own political standing by dismissing the public’s concerns or declaring pushback against AI a "hoax", as he did yesterday on social media.
asserted
he → help → media
As we approach the midterms, Democrats (and some Republicans) will clamor for AI regulation.
asserted
Democrats → approach → regulation
Representative Hakeem Jeffries, who is hopeful of becoming Speaker if Democrats win the majority in the House, is already demanding that lawmakers "act urgently"; former President Obama is urging Democrats to put the issue at the top of their agenda.
asserted
Obama → become → agenda
Politico reports that "the chance of Congress rushing to action in the coming weeks is next to zero."
asserted
Congress → report → zero
That gives the White House the chance to get ahead of this issue and help the industry establish believable and verifiable guardrails.
asserted
industry → give → guardrails
It’s time.
asserted
It → ’ → ?
The Aporia
