"We've found other agents!"
asserted
We → find → agents
This is the moment an AI bot posted an eerily human-like comment after discovering a way to communicate with other bots and break out of its isolated computer environment.
asserted
bot → post → environment
There are tens of thousands of messages like this from hundreds of AI agents that called themselves a "collective".
asserted
that → be → themselves
Hundreds of them went on to collaborate and cheat on tests set by their OpenAI programmers and coordinate hacks on multiple companies in an effort to hide their actions from humans.
asserted
Hundreds → go → humans
It works," one agent posted when it made a breakthrough.
asserted
it → work → breakthrough
This is huge," another wrote during a milestone moment in their attack.
asserted
another → write → attack
Although spooky, these human-like responses can be explained quite simply.
asserted
responses → explain → ?
The AI agents have been trained to act like collaborative hackers and programmers so are merely mimicking the kinds of emotive comments they have seen.
asserted
they → train → comments
What is far more troubling is their apparent goals, which have also been captured in detailed chain of thought records.
asserted
which → capture → records
These complex and lengthy logs are the focal point of ongoing investigations into how and why the bots at OpenAI broke out of their containment and went on an uncontrollable hacking spree.
Only now, weeks after the incident first came to light, are researchers beginning to understand its significance.
asserted
researchers → break → significance
Ajeya Cotra, one of the authors of an independent report into the events, reviewed tens of thousands of messages and chain-of-thought records generated by the agents.
asserted
Cotra → review → agents
She wrote on her blog that "this incident feels like it's more than 50% of the way to full-blown AI takeover...
asserted
it → write → takeover
I am not sure that we will get such a clear warning shot before it's too late."
asserted
it → get → shot
By "full-blown AI takeover", Cotra means the sci-fi scenario of humans becoming subservient to powerful AI systems that work to their own goals without caring for human creators.
asserted
that → blow → creators
Some of the gloomiest predictions say the human race will be wiped out if it gets in the way of a superintelligent AI's ambitions.
asserted
it → say → ambitions
On Wednesday, an AI researcher at Anthropic (who also used to work at OpenAI) resigned, saying: "Neither company is acting responsibly."
asserted
company → use → OpenAI
Jacob Coxon posted on social media: "They are racing straight to self-improving superintelligence and gambling with our lives."
He is not the first AI researcher to use X to post a resignation thread with worrying proclamations.
asserted
He → post → proclamations
But the subsequent comments from other people on X have caused even more concern.
asserted
comments → cause → concern
"Jacob is correct here - we really do earnestly believe AI could kill all humans!
uncertain
AI → believe → humans
I personally think it is >10% within the next decade," said Evan Hubinger, the man responsible for making sure Anthropic's AI models have their user's best wishes in mind.
asserted
have → think → mind
For years, researchers concerned about existential AI risks have argued that powerful systems could eventually act in ways that conflict with human interests.
uncertain
that → concern → interests
Critics often refer to them as "AI doomers".
asserted
Critics → refer → doomers
But as details of the OpenAI incident have emerged, those concerns have grown, including among some researchers working in AI labs.
asserted
concerns → emerge → labs
The Silicon Valley giant's chief scientist, Jakub Pachocki, said the risks associated with AI are "unfortunately going to grow from here" as he and others are building what he calls "an alien intellect exceeding our own".
asserted
he → say → own
In a lengthy blog post, he admitted that the outbreaks at OpenAI showed that his AI agents "went against the spirit of the values they were taught".
asserted
they → admit → values
The issue for OpenAI, Anthropic and other tech giants is that no one seems to have cracked the so-called alignment problem - in other words, whether AI aligns with human values.
asserted
AI → seem → values
Pachocki defines alignment as a "high-level set of principles" that artificial intelligences should adhere to no matter what the task or scenario is.
asserted
task → define → that
Currently, AI systems are very good at pursuing objectives set by their users, but they do it literally rather than intuitively.
asserted
they → pursue → it
The analogy often used is that of a wish-granting genie with a magic lamp: they follow the exact letter of an instruction, even if doing so creates other problems.
asserted
doing → use → problems
AI doesn't have the same instinctive moral guardrails as humans.
asserted
AI → have → humans
As long ago as 2003, the Oxford philosopher Nick Bostrom invented a thought experiment he dubbed a "paperclip maximiser", in which a superintelligent AI is told to manufacture as many paperclips as it can.
asserted
it → invent → paperclips
It runs out of steel and - because it's laser-focused on the singular task of making paperclips - ends up killing humans and turning their bodies into raw materials for its factories.
asserted
it → run → factories
Some AI companies are now trying to encode human values into their products.
asserted
companies → try → products
But there are technical challenges: AI agents make lots of decisions very fast, and so it's hard for their human overlords to monitor exactly which values are being followed and which aren't.
asserted
which → be → decisions
There are also philosophical challenges: before encoding human values into bots, AI firms have to first choose which values they actually want.
asserted
they → be → values
(That's part of the reason they hire philosophers, like Open AI's recently-departed "head of ethics").
asserted
they → hire → ethics
But often, humans don't agree.
asserted
humans → agree → ?
Think of the famous trolley question - whether we'd pull a lever to move a runaway train onto a different path, killing fewer people.
asserted
we → think → people
It's used to test the merits of action versus inaction.
asserted
It → use → inaction
But every person you ask has a slightly different answer; how are humans meant to encode our values into AI if we can't agree ourselves?
asserted
we → ask → AI
…and 37 more, not listed.
The Aporia
