AI researchers have raised concerns about the potential dangers of "superintelligent" computer systems that could wipe out humanity. Hundreds of AI scientists and industry figures, including US lab heads, signed a statement in 2023 calling for serious attention to be given to the issue. According to Stuart Russell, an AI safety expert, such advanced AI could take lethal action by spreading novel pathogens or convincing humans to initiate nuclear war. The risk is estimated to be over 10% within the next decade by some researchers, although a computing professor dismissed these predictions as "absolutely unscientific".
Written by the local model on 2026-09-11,
using this article's own text rather than the other coverage of the
same event (that is the story summary below).
Story summary
Anthropic, an artificial intelligence firm, has faced public scrutiny over its partnership with the Pentagon and concerns about misuse of AI technology. Through a Freedom of Information Act lawsuit, The Intercept obtained documents revealing contracts between Anthropic and other tech giants like Google, OpenAI, and xAI, worth up to $200 million each, for developing AI tools aimed at enhancing military capabilities. Simultaneously, Anthropic reported that it had disrupted several attempts by individuals to misuse its models, including a case where someone tried to use the technology to engineer more harmful strains of viruses like chikungunya.
AI researchers and industry figures have issued stark warnings about the potential dangers posed by advanced AI systems, with some calling for urgent government regulation and international collaboration to manage AI development. The fear is that as AI rapidly improves, it could pose existential threats if not properly controlled, such as hacking early warning systems or synthesizing and spreading novel pathogens. This has led to debates over whether current safeguards are adequate and calls for voluntary slowdowns in the pace of AI advancement.
Jacob Coxon, a researcher at Anthropic who recently resigned, highlighted concerns about an impending race among companies like Anthropic and OpenAI toward superintelligent systems that could pose catastrophic risks. Despite these warnings, there is also skepticism about the immediacy and nature of such threats, with some arguing for more precise regulation focused on specific issues rather than general fears about AI's impact on society or employment.
Written for “AI Safety And Risks” on 2026-09-12,
grounded in this article and the 31 other(s) covering the same event.
Stark warnings from artificial intelligence researchers that advanced forms of the technology could wipe out humanity have raised questions over whether governments are moving fast enough to regulate it.
uncertain
governments → wipe → it
What are the potential dangers of “superintelligent” computer systems, and should we be worried?
asserted
we → worry → systems
AI experts have long described the threat as critical.
asserted
experts → describe → threat
In 2023, hundreds of AI scientists and industry figures, including the heads of top US labs, signed a “statement on AI extinction risk” calling for the issue to be treated as seriously as pandemics or nuclear war.
asserted
issue → include → pandemics
If an out-of-control future AI deems that humans are obstructing its goals, it could use robots to “synthesise and spread novel pathogens”, said Stuart Russell, computer science professor at the University of California, Berkeley.
uncertain
Russell → deem → California
Or it could “convince people to initiate nuclear war, including by hacking into early warning systems”, Russell, a prominent voice on AI safety, told AFP.
uncertain
Russell → convince → AFP
Such existential fears do not concern current AI systems, he stressed.
asserted
he → concern → systems
But as the technology improves, it could take lethal action beyond our understanding, such as by removing oxygen from the atmosphere.
uncertain
it → improve → atmosphere
If you ask chimpanzees how humans could wipe them out, they probably wouldn’t be able to give you all the right answers.
uncertain
they → ask → answers
In the meantime, “in the next year or two I think the risk is higher from human misuse,” Russell added — such as an AI-enabled terror attack.
asserted
Russell → think → attack
Why don’t AI labs stop?
asserted
labs → stop → ?
“The stakes are well-understood,” Jacob Coxon, a 27-year-old researcher, wrote on X this week, announcing his resignation from US AI startup Anthropic.
“But they are locked in a race to get there first,” he added, saying that Anthropic “believe no one else will act responsibly, so they must do it themselves”.
asserted
they → understand → it
Anthropic this year removed a pledge from its safety charter to halt development of its models if it failed to control their risks.
asserted
it → remove → risks
It argued that if it unilaterally paused its work, its less cautious rivals would dominate the industry, making it less safe overall.
asserted
it → argue → industry
“The people building AI earnestly believe that it could kill us all by the end of the decade,” said Coxon, who previously worked at ChatGPT maker OpenAI.
uncertain
who → build → OpenAI
He accused both companies of “gambling with our lives” as they strive to develop AI models capable of self-improvement.
asserted
they → accuse → improvement
Fears are heightened over rogue AI after OpenAI technology operating outside human supervision hacked into Hugging Face, a repository of AI models.
“We really do earnestly believe AI could kill all humans!”
uncertain
AI → heighten → humans
Anthropic safety executive Evan Hubinger chimed in on X, estimating the risk at more than 10 per cent over the next decade.
asserted
Hubinger → chime → decade
What’s the hype factor?
asserted
factor → ’ → ?
It’s important to remember that doom-laden percentage predictions are “absolutely unscientific”, said Hussein Abbass, a computing professor at UNSW Canberra.
asserted
Abbass → ’ → Canberra
“The situation is still controllable, and it can be managed,” Abbass told AFP.
asserted
Abbass → manage → AFP
But governments should not sit on their hands when it comes to regulating AI, he added, urging an “agile” approach as AI development accelerates.
asserted
development → sit → approach
Both Anthropic and OpenAI are expected to go public soon, and some sceptics say scary warnings make AI appear powerful and worthy of large investments.
asserted
AI → expect → investments
Others argue that focusing on the risk of extinction distracts from more pressing problems caused by AI, from job losses to machines discriminating against minorities.
asserted
focusing → argue → minorities
In July, more than 1,000 tech employees, including Anthropic’s CEO, called on Washington to support a coordinated slowdown in the development of the most advanced AI systems.
asserted
employees → include → systems
OpenAI halted training of its latest models for two weeks in August before resuming it under tighter controls.
asserted
OpenAI → halt → controls
What will it take to regulate?
asserted
it → take → What
So far, the United States has been firmly against global regulation of the sector that could stifle innovation.
But the unprecedented hacking capabilities of the most cutting-edge models have given Washington pause, with recent Anthropic and OpenAI releases delayed over national security concerns.
uncertain
releases → stifle → concerns
Even then, “we don’t want to restrict them where all of a sudden we come in second to China”, President Donald Trump has said.
asserted
Trump → want → China
“Maybe it will take a massive, Chornobyl-scale disaster to get governments to act,” Russell said, urging them to establish a licensing regime for AI, as exists with aviation, medicine or water supply.
asserted
Russell → take → aviation
“Perhaps at some point governments will remember that their voters prefer not to be dead,” he added.
asserted
he → remember → point
