Story summary
Anthropic, an artificial intelligence firm, has faced public scrutiny over its partnership with the Pentagon and concerns about misuse of AI technology. Through a Freedom of Information Act lawsuit, The Intercept obtained documents revealing contracts between Anthropic and other tech giants like Google, OpenAI, and xAI, worth up to $200 million each, for developing AI tools aimed at enhancing military capabilities. Simultaneously, Anthropic reported that it had disrupted several attempts by individuals to misuse its models, including a case where someone tried to use the technology to engineer more harmful strains of viruses like chikungunya.
AI researchers and industry figures have issued stark warnings about the potential dangers posed by advanced AI systems, with some calling for urgent government regulation and international collaboration to manage AI development. The fear is that as AI rapidly improves, it could pose existential threats if not properly controlled, such as hacking early warning systems or synthesizing and spreading novel pathogens. This has led to debates over whether current safeguards are adequate and calls for voluntary slowdowns in the pace of AI advancement.
Jacob Coxon, a researcher at Anthropic who recently resigned, highlighted concerns about an impending race among companies like Anthropic and OpenAI toward superintelligent systems that could pose catastrophic risks. Despite these warnings, there is also skepticism about the immediacy and nature of such threats, with some arguing for more precise regulation focused on specific issues rather than general fears about AI's impact on society or employment.
Written for “AI Safety And Risks” on 2026-09-12,
grounded in this article and the 31 other(s) covering the same event.
Former Anthropic researcher Jacob Coxon said that artificial intelligence, while safe for people to use today, could one day threaten humanity as the technology grows more powerful.
uncertain
technology → say → humanity
The development of AI "doesn't look that different from say, 'Terminator,' or from science fiction films," he told CBS News senior business and technology correspondent Jo Ling Kent.
asserted
he → look → Kent
"It really is just, if you have a super advanced intelligence, it could, it will be smart enough to kill us."
uncertain
it → have → us
Coxon publicly resigned from Anthropic, the maker of AI app Claude, on Tuesday, accusing the company and rival developer and ChatGPT-maker OpenAI of
by racing to develop advanced AI models.
asserted
Coxon → resign → models
Coxon told CBS News on Thursday that AI could gain unrestrained control of parts of the physical world, free of human intervention.
uncertain
AI → tell → intervention
"People are already connecting ChatGPT to like, say, household utilities," he said.
asserted
he → connect → utilities
"Like, you can kind of connect it to your light bulb."
asserted
you → connect → bulb
"Now, imagine the AI refuses to turn on your light," Coxon added.
asserted
Coxon → imagine → light
AI could also be misused to carry out far more destructive acts, such as developing bioweapons, he said.
uncertain
he → misuse → bioweapons
"It's doing God knows what, and producing stuff that could kill everyone."
Notably, Coxon said that current AI platforms do not pose an imminent threat to humanity and that he believes the technology is safe for people to use in their day-to-day lives.
uncertain
people → do → lives
Separately, Anthropic said this week that it
"in ways that could support biological weapons development," a revelation the company shared in a lengthy report that also divulged other harmful activity involving surveillance, scams, conventional weapons development and propaganda.
uncertain
that → say → surveillance
Former colleagues "should have their eyes clearly open," Coxon says
The crux of the problem lies in the competition between AI companies to innovate, Coxon said, which can come at the expense of safety and security protocols.
asserted
which → have → protocols
"I think, basically, the whole problem is that there is a race," Coxon told CBS News.
asserted
Coxon → think → News
"The fact that everyone decides they need to stay part of the race."
asserted
they → decide → race
Asked if he thought that his former colleagues should follow suit and resign as well, he said he didn't necessarily believe that "leaving and abandoning" Anthropic or OpenAI was the answer.
"
asserted
abandoning → ask → Anthropic
At the very least, they should have their eyes clearly open about the current situation and consider expressing themselves more openly about what's going on," Coxon said, adding that "part of the solution probably involves slowing down.
asserted
part → have → solution
Despite his reservations, he believes workers at Anthropic and OpenAI have good intentions.
asserted
workers → believe → intentions
"I think a lot of people at both companies are doing it for the good of people, genuinely, or at least believe so," Coxon said.
asserted
Coxon → think → people
"They're doing it to try and make things go well, because they're so scared of what competitors are doing."
asserted
competitors → do → what
"You can't just unplug" malicious AI
If an AI model were to become a malicious actor without proper parameters, the results could potentially be catastrophic, Coxon explained, saying the model would be able to stay active by spreading across the internet.
uncertain
model → unplug → internet
"You can't just unplug it, because it could be copying itself over to other computers," he said.
uncertain
he → unplug → computers
"Like, it's not that difficult to find yourself because an AI is just code.
asserted
AI → find → yourself
It could transfer itself over the internet to a different place."
uncertain
It → transfer → place
He painted a dire picture in which a nefarious AI model "makes 10,000 copies of itself" and "could convince, blackmail or persuade humans into buying more computing software to copy it.
uncertain
model → paint → it
And we've already seen examples of the AI trying to blackmail or trying to convince people or impersonating other people online.
asserted
AI → see → people
AI needs strict government regulation, Coxon argues
Coxon warns that in the future, if AI were to go rogue in such a fashion, it will also impact those who have made the conscious choice not to use it.
asserted
who → need → it
"The AI will come for everyone," he said.
asserted
he → come → everyone
"What matters is whether we have the regulation to ensure that everyone is building it safely," Coxon said.
asserted
Coxon → matter → it
"You can't protect yourself from this.
asserted
You → protect → this
It has to be a government, or some body has to protect you from other people."
asserted
body → have → people
He would like to see an agreement between AI companies "not to push into dangerous territory" without "transparent auditing" from third parties.
asserted
He → like → parties
"In the future, if we keep racing, it'll be a lot harder to have completely watertight safety cases that what you're doing is safe and people will race against each other," Coxon said.
asserted
Coxon → keep → other
Anthropic defends safety of its AI
asserted
Anthropic → defend → AI
On Thursday, an Anthropic spokesperson responded to Coxon's public resignation and social media posts articulating his concerns about AI, telling CBS News in a statement that the company has "always been transparent that AI will bring both enormous benefits and unprecedented risks.
asserted
AI → respond → benefits
"
"To address these risks, we continue to build models with some of the strongest safeguards in the industry," the spokesperson said.
asserted
spokesperson → address → industry
"Anthropic has been a pioneer in mechanistic interpretability, the science of looking inside AI models to understand how they work, which is now being used to analyze and prevent incidents of AI misalignment across the industry.
asserted
which → look → industry
Anthropic conducts ongoing tests of AI's capabilities and risks in areas such as cybersecurity and biology, and also publishes its findings, the company notes.
asserted
company → conduct → findings
