- Published
An artificial intelligence "kill switch" which can be checked by a third party may need to be mandatory for companies, a co-founder of one of the world's largest AI firms has told the BBC.
uncertain
founder → publish → BBC
Jack Clark, one of seven founders of Anthropic, said a way of shutting off AI software completely if it gets too dangerous was something society "might want to eventually pass rules around".
uncertain
society → say → rules
Clark said "most labs have different ways of being able to pull the plug", including Anthropic, but said lawmakers may need to enforce having one.
AI's rapid development and fears over the risks it poses to humanity have been thrust into the spotlight by a series of warnings from executives and staff at AI firms.
uncertain
it → say → firms
Some have said publicly there is a chance the technology could, if unchecked, kill all humans.
uncertain
technology → say → humans
Anthropic chief executive Dario Amodei over the weekend called for the pace of AI development to slow and be more closely monitored, as the company has done before, though some have questioned the motivations behind this.
asserted
some → call → this
Amodei added that any action to rein in AI development should be done "without sacrificing commercial advantage".
asserted
action → add → advantage
Clark told the BBC that specifics around "kill switch" requirements and verification should be part of "the larger policy conversation" taking place around AI.
asserted
specifics → tell → AI
"Should you mandate for companies to definitely have a kill switch?
asserted
companies → mandate → switch
Is that kill switch verifiable by a third party?" he asked.
asserted
he → verifiable → party
"I think that's the kind of thing society is going to want to know and might want to eventually pass rules around."
uncertain
society → think → rules
Anthropic, which was formed in 2021 by a group of former employees of its rival OpenAI, is currently at the centre of a debate around AI safety.
asserted
which → form → safety
Last week, a post from an artificial intelligence researcher who quit Anthropic over concerns AI could wipe out humanity went viral.
uncertain
AI → quit → humanity
In response, Anthropic scientist Evan Hubinger said he personally thought the possibility of human extinction from AI was ">10% within the next decade".
asserted
possibility → say → decade
Computer scientist and Nobel Prize winner Geoffrey Hinton, known as the "Godfather of AI", told the BBC on Friday that a 10% chance of AI killing all humans was "not unreasonable", external.
asserted
AI → know → humans
He and others with similar concerns have suggested AI could do so by taking control of important systems connected to the internet and turning them on humans.
uncertain
AI → suggest → humans
However, some in the AI industry have suggested the fears around it destroying humanity are overstated or may be designed to generate hype.
uncertain
it → suggest → hype
Clement Delangue, leader of the developer platform Hugging Face which was hacked by OpenAI bots, said last week: "Sorry, but asking Jacob [Coxon] about AI extinction risk is like asking your AC guy about climate change."
asserted
asking → hack → change
"Not saying it's necessarily uninteresting or wrong per se but let's keep things in perspective."
asserted
's → say → perspective
George Arison, the leader of Grindr, said the fears going around AI tools were being used by the companies to support their business plans.
asserted
fears → say → plans
Anthropic, which was valued at $965bn (£713bn), is preparing for an initial public offering on the stock market, allowing people to buy shares in its firm.
asserted
people → value → firm
OpenAI, which was most recently valued at $852bn (£630bn), had been expected to do the same, but OpenAI's Altman said on Friday this would not happen this year due to the current debate around AI safety.
asserted
this → value → safety
Arison said: "The only way to justify these valuations is to actually claim: 'I'm going to take over every industry and I'm going to take over every job, and my AI is going to be doing all that work.'
uncertain
AI → say → work
"
Asked what percentage he would put on all humans being killed by AI, Clark said: "I don't think these statistics are that useful", but added that allowing AI to continue as a "totally unregulated industry" was a bad idea.
asserted
AI → ask → industry
"We are rolling dice with immense risks," Clark said.
asserted
Clark → roll → risks
"And the point is, we have to change the course of this industry."
asserted
we → have → industry
US lawmakers have put forward legislation dubbed the Kill Switch Act that would require companies to have a way to shut down problematic AI tools.
asserted
that → put → tools
It would also give certain government agencies the power to demand a tool be turned off or limited.
asserted
tool → give → power
However, US President Donald Trump has rejected calls to slow the development of AI, saying on social media "AI taking over the World, destroying Humanity, and all other things bad, is a HOAX".
asserted
taking → reject → Humanity
Trump later compared fears to concerns over the "Global Warming Scam of not so long ago, where everyone was going to die from extreme heat".
asserted
everyone → compare → heat
In an earlier post, he wrote: "There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China.
asserted
that → write → it
In the UK, the government has recently rejected the idea of creating a kill switch, with a spokesperson saying it "would not prevent them being developed or misused elsewhere".
asserted
them → reject → switch
Anthropic created the popular chatbot Claude and has this year released a number of increasingly capable AI models, the technology underlying AI chatbots.
asserted
Anthropic → create → chatbots
Alongside OpenAI, Anthropic has since self-reported a number of incidents where AI agents, which are bots that operate somewhat autonomously, have acted in ways that were unexpected.
asserted
that → report → ways
The Aporia
