OpenAI and Anthropic agree: AI safety and "alignment" concerns have been brewing for a long time, but they reached a fever pitch last week as several Anthropic employees' tweets went viral.
asserted
tweets → agree → pitch
"I resigned from Anthropic today," wrote Jacob Coxon.
asserted
Coxon → resign → Anthropic
"I spent the last three years doing pretraining research at both OpenAI and Anthropic.
asserted
I → spend → OpenAI
Neither company is acting responsibly.
asserted
company → act → ?
They are racing straight to self-improving superintelligence and gambling with our lives."
"Do not underestimate the power of this technology," he added.
asserted
he → race → technology
"These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.
asserted
that → hack → power
We have all witnessed the progress in each of these domains, and progress is not slowing.
asserted
progress → witness → domains
"
The Reason Roundup Newsletter by Liz Wolfe Liz and Reason help you make sense of the day's news every morning.
asserted
you → help → news
"Jacob is correct here—we really do earnestly believe AI could kill all humans!" wrote Evan Hubinger, another Anthropic worker (with a questionably-placed exclamation point).
uncertain
Hubinger → believe → point
"I personally think it is >10% within the next decade.
asserted
it → think → decade
I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
asserted
we → believe → track
Drake Thomas, an Anthropic safety employee, said he would "burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive."
asserted
we → say → situation
I've described, over the last few weeks, how swarms of rogue agents have acted against developer intentions and stolen data from other companies.
asserted
swarms → describe → companies
Meanwhile, both OpenAI and Anthropic are preparing to go public sometime soon.
asserted
OpenAI → prepare → ?
In an essay released this weekend (called "We Must Pace the Frontier"), Anthropic CEO Dario Amodei calls for a global slowdown of AI development, citing such risks as "losing control of AI systems" and "misuse of AI for cyberattacks and bioterrorism.
asserted
Amodei → release → cyberattacks
"
"Along with my co-founders and employees, I have grappled with this duality of risk and benefit since the beginning of Anthropic," writes Amodei.
asserted
Amodei → grapple → Anthropic
"Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless."
asserted
building → build → it
More prudence is necessary, he added; recursive self-improvement—when AI systems design or train their own successors—must be "pursued very carefully, if at all."
asserted
systems → add → successors
In other words, he wants a development slowdown for all large AI system builders.
asserted
he → want → builders
Amodei's statement was quickly cosigned by OpenAI head Sam Altman as well as Google DeepMind's Demis Hassabis and xAI's Elon Musk.
asserted
statement → cosign → Altman
Meanwhile, former President Barack Obama privately urged House Minority Leader Hakeem Jeffries (D–N.Y.) to develop a plan for AI regulation and urged Democrats to develop a clear plan for how AI ought to be regulated ahead of the 2028 presidential election.
asserted
AI → urge → election
"Stop pretending you need anyone else's permission," responded venture capitalist David Sacks.
asserted
Sacks → pretend → permission
"Stop pretending antitrust law has to be suspended so you can form a cartel.
asserted
you → pretend → cartel
Stop pretending you need a regulatory approval process that supersedes product liability.
asserted
that → pretend → liability
Stop pretending METR [the nonprofit Model Evaluation & Threat Research] is independent when it is intertwined with Anthropic's investors and staff.
asserted
it → pretend → investors
Stop pretending you need those same evaluators to police competitors who aren't even at the frontier.
asserted
who → pretend → frontier
"
"Most of all," he adds, "stop pretending the motivation to slow down is purely altruistic.
asserted
motivation → add → all
You face massive product-liability exposure if your products enable a truly damaging cyberattack.
asserted
products → face → cyberattack
The market already punishes models that behave in unpredictable or unauthorized ways.
asserted
that → punish → ways
After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability.
asserted
OpenAI → trade → reliability
Call it alignment if you want.
asserted
you → call → it
It is also just giving customers what they want.
asserted
they → give → what
"Over the last three years, Jewish institutions across New York City have added millions more to their budgets for security protections amid an increase in antisemitic attacks, rabbis and Jewish community leaders say," reports The New York Times.
asserted
Times → add → attacks
"They have paid to hire more private security guards, increase the number of safety trainings at synagogues and schools, update their security cameras and fortify their buildings.
asserted
They → pay → buildings
They are also relying on more volunteer security guards at temples and encouraging those who attend services to be especially watchful.
asserted
who → rely → services
Their efforts have come into focus ahead of the 10-day period of somber introspection that takes place between Rosh Hashana and Yom Kippur, the holiest holiday in Judaism.
asserted
that → come → Judaism
"Leaders of the John F. Kennedy Center for the Performing Arts, installed by President Donald Trump, say the center is on the brink of bankruptcy and argue that putting the president's name on its facade—and thus ensuring his fundraising efforts—may be the only way to avoid imminent and 'certain fiscal collapse,'" reports The Washington Post.
uncertain
Post → instal → collapse
- "New Geran drones and Banderol cruise missiles are lethal reminders that Russia still has the resources and supply chains to field fresh threats, this time packed with advanced electronics that allow them to find targets accurately and evade interception," reports The Wall Street Journal.
asserted
Journal → field → interception
This "was enabled in large part by components sourced from China, in violation of international sanctions, say Ukraine and others, based in part on wreckage from the few shot down so far."
asserted
Ukraine → enable → few
- A new drug, cychlorphine, is believed to be 10 times as potent as fentanyl.
asserted
drug → believe → fentanyl
…and 3 more, not listed.
