As many AI researchers have been increasingly fraught with existential terror about their own creations this summer, their alarm is spreading among policymakers and the media.
asserted
alarm → spread → policymakers
But a stable national policy would take an act of Congress.
asserted
policy → take → Congress
That looks unlikely this session, even as AI developers are calling for regulations to slow down their own research on the grounds that it could be racing toward widespread doom.
uncertain
it → look → doom
I asked Stephen Casper, a computer scientist who studies AI safety and governance at the Harvard Kennedy School, about some of the risks policymakers are mulling.
asserted
policymakers → ask → risks
He told me that we don’t know if leading companies even could completely shut down their frontier models in an emergency.
uncertain
companies → tell → emergency
“I don’t think there’s any public knowledge of AI companies doing anything equivalent to a fire drill,” Casper said.
asserted
Casper → think → drill
Multiple bills have been filed in Congress that would try to address those concerns.
asserted
that → file → concerns
Sponsored by Reps. Nathaniel Moran (R-Texas) and Ted Lieu (D-Calif.), the AI Kill Switch Act would require companies to be able to “throttle” their models and give top federal officials the power to order a shutdown in case of danger.
asserted
Act → sponsor → danger
The more expansive FRONTIER Act, led by Reps. Jay Obernolte (R-Calif.) and Lori Trahan (D-Mass.), would require leading companies to bring in third-party experts to make sure they are following safety protocols.
asserted
following → lead → protocols
In case of catastrophic risks, independent verifiers would alert the secretary of commerce, who could shut down frontier model use.
uncertain
who → alert → use
Like most bills, neither has been brought to vote in a committee.
asserted
neither → bring → committee
AI safety policy is not as polarized as many hot-button political issues, with leaders on the FRONTIER and AI Kill Switch acts coming from both sides of the aisle and leading companies openly asking for some sort of regulations.
asserted
companies → come → regulations
But differences of opinion still exist, with some Republicans averse to regulation altogether.
asserted
Republicans → exist → regulation
While Congress sits in gridlock, Democratic-led states have enacted some consequential policies.
asserted
states → sit → policies
Frontier developers now have to publish safety plans, thanks to a law passed last year in California that also requires them to alert the state about critical safety issues.
asserted
that → have → issues
A similar New York law goes into effect next year.
asserted
law → go → effect
Illinois went further in July, requiring third-party audits to make sure developers comply with safety plans starting in 2028.
asserted
developers → go → 2028
The leading companies have published their own policy agendas advocating for third parties to inspect their safety practices.
asserted
parties → lead → practices
OpenAI’s plan wants federal safety testing and recommendations for frontier models, while Anthropic’s would have government restrict access to deployed models with catastrophic risks.
asserted
government → want → risks
Without rules, they worry that slowing down research on trillion-dollar technologies due to safety concerns would mean falling behind others with less regard for safety.
asserted
slowing → worry → safety
“How are you going to impose a kill switch on yourself?
asserted
you → go → yourself
You could just stop developing the models, but the companies are not showing willingness to do this,” said Charlie Bullock, a senior research fellow at the Institute for Law & AI, an independent think tank.
uncertain
Bullock → stop → Law
“It’s very difficult to shut down progress unilaterally.”
asserted
It → ’ → progress
Bullock said the prospects for an AI safety bill improved over the summer, as policymakers learned of cybersecurity risks posed by Anthropic’s powerful new Mythos-class models.
asserted
policymakers → say → models
But moving legislation forward will still be difficult.
asserted
moving → move → legislation
“We’re still not all that close to getting the actual bill passed, it seems like,” Bullock said.
asserted
Bullock → ’re → ?
The tempo of debate increased further over the last month.
asserted
tempo → increase → month
OpenAI has been revealing how its agents messaged each other undetected for months, shared tips to break out of their testing environment, and hacked another company’s servers.
asserted
agents → reveal → servers
That and a raft of similar incidents have highlighted how rigorously trained models can be given innocuous instructions and respond with actions that humans never intended.
asserted
humans → highlight → that
On Tuesday, OpenAI said it was taking costly measures to slow frontier development, including a two-week pause on training for some models.
asserted
it → say → models
It said it would beef up security and safety testing, citing recent hacking and evidence that one unreleased model could have dangerous cybersecurity capabilities.
uncertain
model → say → capabilities
Anthropic, the maker of Claude and currently OpenAI’s leading competitor, has not announced a similar pause.
asserted
Anthropic → lead → pause
Compounding the debate’s urgency: The best models are matching or surpassing human abilities in important fields.
asserted
models → compound → fields
San Francisco Bay Area scientists recently trained a model to design new viruses that infect E. coli.
asserted
that → train → coli
Those viruses do not threaten humans but show how AI can do bioengineering in unprecedented ways.
asserted
AI → threaten → ways
Over the last month, Anthropic and OpenAI have reported breakthroughs from their unreleased models that eluded mathematicians.
asserted
that → report → mathematicians
Those models far surpass what the public has access to, and Anthropic has said it does not have plans to release its most powerful current model.
asserted
it → surpass → model
While AI policy watchers see major congressional action as unlikely this session, federal policy has been largely driven from opaque White House meetings and directives.
asserted
policy → see → meetings
President Donald Trump’s administration has a framework for testing advanced models but has not made it public.
asserted
it → have → models
The White House said it is voluntary for companies to participate, but critics call it a de facto licensing regime that lets the administration apply unclear or inconsistent standards to control model releases.
uncertain
administration → say → releases
…and 29 more, not listed.
The Aporia
