On Saturday, September 12, Dario Amodei, the chief executive officer of the artificial intelligence company Anthropic, published a 3,800-word essay calling for AI companies to slow down their improvement of AI models.
asserted
companies → publish → models
Amodei expressed concern that AI models are themselves pushing advances faster than engineers can understand them.
asserted
engineers → express → them
He noted that July’s OpenAI–Hugging Face incident, in which programs designed to hack into systems found weaknesses that permitted them to escape the “sandbox” in which designers were testing them for about a week before anyone noticed had, luckily, been relatively harmless, but warned that “in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage.”
uncertain
that → note → damage
Within a year, he warned, such a swarm could take over the entire internet, causing hundreds of billions of dollars in damage.
uncertain
swarm → warn → damage
In his essay, Amodei called for AI companies to commit to giving embedded third-party evaluators access to their work, rather like the regulatory supervisors in banking.
asserted
companies → call → banking
He called for AI companies in democratic countries to establish safety standards and limit their rate of progress.
asserted
companies → call → progress
And he called for “[t]he US and other democratic governments [to] attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.”
asserted
this → call → compliance
Today on Face the Nation, Amodei explained to host Jo Ling Kent: “My view here is it has always been very strange that this technology is being built by a private company.
asserted
technology → explain → company
People ask me that question all the time.
asserted
People → ask → question
Why isn’t this being built by government?
asserted
this → build → government
And the strangest thing about it is, I agree with them.
asserted
I → agree → them
I’m uncomfortable.
asserted
I → ’m → ?
Government didn’t build this technology.
asserted
Government → build → technology
This company, this technology came from the private sector, and we—Anthropic, and I would hope other companies, have done everything we can to try to have legitimate oversight mechanisms.
asserted
we → come → mechanisms
“I think the government and the public need to have a stake.
asserted
government → think → stake
And this is why we’ve supported regulation of the technology.
asserted
we → support → technology
Regulation constrains the private companies.
asserted
Regulation → constrain → companies
Regulation allows the public and its elected representatives to have a say, and limits what the private companies can do.”
asserted
companies → allow → what
Asked if he would be willing to give the technology itself to the government, Amodei answered that he might be willing to give it to the right combination of governments.
uncertain
he → ask → governments
“I want to be very clear about this,” he said.
asserted
he → want → this
“I am concerned that one single government could abuse this technology just as easily as a single company could.
uncertain
company → abuse → technology
But I think a combination of democratically elected governments—I don’t know about hand over, but some kind of oversight, some kind of joint governance.
asserted
I → think → governance
Again, that would be the work of years, but I wonder if that’s the direction we need to go in.”
asserted
we → wonder → years
As Mike Isaac of the New York Times noted, Amodei’s call came days after an Anthropic researcher, Jacob Coxon, resigned, posting on social media that neither OpenAI nor Anthropic was “acting responsibly.
asserted
OpenAI → note → media
They are racing straight to self-improving superintelligence and gambling with our lives.”
asserted
They → race → lives
They are, he said, building “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
asserted
that → say → power
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon said.
uncertain
Coxon → build → decade
He claimed that people at OpenAI might not yet realize what they’re constructing, while those at Anthropic do understand but believe “they are locked in a race to get there first—they believe no one else will act responsibly, so they must do it themselves, despite the risk.
uncertain
they → claim → risk
Critics are skeptical, suggesting that AI leaders are hyping their product to make it appear more valuable than it really is.
asserted
it → suggest → product
There is widespread concern that the massive spending on AI infrastructure like data centers and chips might not deliver the revenue and profits that would justify such spending.
uncertain
that → be → spending
In June, Kate Brennan, associate director of independent research institute AI Now, told Aimee Picchi of CBS News: “The returns are not coming in, and the claims that are being made, in terms of efficiency or productivity numbers, are not netting out.”
uncertain
that → tell → numbers
Others wonder if Amodei’s public concerns aren’t designed to get the government to regulate AI in such a way that it creates a framework that would make it hard for smaller companies to break into the market.
asserted
companies → wonder → market
Still, Kate Conger of the New York Times noted last week that in July, more than 1,300 employees from Anthropic, OpenAI, Meta, and Google’s DeepMind—all leading AI companies—signed an open letter asking the U.S. government to regulate AI to slow down its development.
asserted
employees → note → development
Conger also notes that in August, more than 100 tech companies offered their models to hospitals and infrastructure systems to enable them to guard against cyberattacks powered by AI.
asserted
companies → note → AI
After Amodei’s essay appeared, Sam Altman, the chief executive officer of OpenAI; Elon Musk, who has been increasing spending on AI through his SpaceX rocket company; and Demis Hassabis, chair of Google DeepMind, all posted their support for slowing down the pace of AI improvements.
asserted
who → appear → improvements
On October 30, 2023, President Joe Biden issued Executive Order 14110, calling for the “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.”
asserted
Biden → issue → Intelligence
The document said AI must be safe and secure.
asserted
AI → say → ?
It called for the promotion of “responsible innovation, competition, and collaboration” and specified that AI must engage U.S. workers and advance equity and civil rights as well as privacy and civil liberties.
asserted
AI → call → equity
The development of AI must protect consumers, it said, and the government must make sure the technology is deployed responsibly.
asserted
technology → protect → consumers
Revoking this executive order was one of the first things Trump did on January 20, 2025.
asserted
Trump → revoke → January
…and 25 more, not listed.
The Aporia
