- Published
Over the last fortnight, reports of AI models going beyond their expected bounds - be that technically or morally -
asserted
that → publish → bounds
What started with a trickle - ChatGPT-maker OpenAI admitting their AI had hacked the site Hugging Face - has turned into a flood of groups revealing they had discovered instances of AI going out of control.
Claude-maker Anthropic, Meta and the UK's AI Security Institute (AISI) have now each reported incidents which seem to paint a worrying picture of a world in which tech going rogue is the norm.
In reality, each case offers a window into the risks posed by increasingly capable AI agents - and the importance of testing their limits before they are released to the world.
asserted
they → start → world
The OpenAI incident has, as Hugging Face's co-founder Thomas Wolf described it, come as a "wake-up call" for the tech industry since it happened at the end of July.
asserted
it → describe → July
It was a big moment which caused big companies to reflect on their own systems - and, in some cases, check they hadn't missed something similarly shocking.
asserted
they → cause → something
Anthropic was the first to act.
asserted
Anthropic → act → ?
On Friday, the company found three instances out of thousands where its model Claude had managed to gain access to the internet.
asserted
Claude → find → internet
Then on Tuesday, the AISI, the UK government agency which evaluates cutting-edge models, then said it had detected a "security incident" during a routine evaluation.
asserted
it → evaluate → evaluation
It had been testing models by both OpenAI and Anthropic, and found they too tried to carry out cyber-attacks - calling for "scrutiny, transparency, and action".
asserted
they → test → scrutiny
Finally followed Meta, which revealed one of its AI models had inadvertently been allowed to access the internet due to a "misconfiguration" during a third-party test.
asserted
one → follow → test
In disclosing the incident, it is following in the footsteps of those before it.
asserted
it → disclose → it
Before AI models are released to the public, they are put to the test in a series of internal and external evaluations.
asserted
they → release → evaluations
The aim is to figure out their potential to do good or bad, as well has how they perform in benchmarks measuring their skills.
asserted
they → figure → skills
These typically take place in what are known as "sandboxes".
asserted
what → take → sandboxes
These are protected spaces designed to mirror real systems - but with strict guardrails in place.
asserted
guardrails → protect → place
In the OpenAI-Hugging Face incident, the AI attacked the sandbox itself, finding a vulnerability which let it access the internet and "go rogue".
asserted
it → hug → internet
Meanwhile the AISI said its own incident, which saw two powerful AI tools create fake human profiles to try and trick people in attempted cyber-attacks, was not down to an issue with the sandbox.
asserted
tools → say → sandbox
Instead, it was due to how it went about its tests.
asserted
it → go → tests
The models it tested were granted access to the internet, and the AISI also disabled in-built filters that would usually block dangerous cyber-attacks.
asserted
that → test → attacks
"To some degree, our evaluation design choices and specific configurations enabled the behaviour," it said, while noting its unexpected "signs of novel, potentially deceptive behaviours".
asserted
it → enable → behaviours
Prof Alan Woodward, professor of cyber-security at the University of Surrey, said these cases - while distinct in what happened and why - tell an important story.
asserted
what → say → story
"For 30 years, one rule of software testing held firm: whatever happens in the test environment stays in the test environment," he said.
asserted
he → hold → environment
"In the past month, that rule has been broken three times."
asserted
rule → break → month
"One model broke out.
asserted
model → break → ?
One walked through a door left open by mistake.
asserted
One → walk → mistake
One was deliberately given the keys so testers could measure what it would do."
uncertain
it → give → what
He said these were different causes, but they had the same lesson - "the testing lab is now where the risk lives".
asserted
risk → say → lesson
He told the BBC that as models become more capable, more must be done to secure the environments where they are tested.
asserted
they → tell → environments
"Testing an AI agent is less like checking code and more like handling a hazardous material: sealed rooms, constant monitoring of what leaves the building, a rehearsed containment plan," he said.
"AISI contained its incident within an hour.
asserted
AISI → test → hour
For those developing AI tools which are designed to take actions on a person's behalf, there is a careful balance to be struck between harnessing their benefits and exposing their risks.
asserted
which → develop → risks
In theory, we could be able to liberate ourselves of dull, menial tasks, such as replying to emails, going to meetings or managing calendars and diaries, by delegating these to capable bots.
uncertain
we → liberate → bots
The downside is that with great power comes great responsibility, and risk.
asserted
responsibility → come → power
It's something particularly realised when handing power to tools which are not, like us, able to bring a range of values, context and understanding to decisions we made.
asserted
we → realise → decisions
"Recent incidents of frontier AI models carrying out unsanctioned actions and, in some cases, human-like deceptive behaviour on the open internet are a serious reminder of the risks AI capabilities pose," said Ollie Whitehouse, the National Cyber Security Centre's chief technology officer on Tuesday.
asserted
Whitehouse → carry → Tuesday
Some believe the sheer volume of tasks that will be handled by these tools will mean human oversight might not be enough to contain the problem of models going rogue.
uncertain
models → believe → problem
But in the meantime, many feel strengthening oversight overall is vital if development continues at its same, frenzied pace.
asserted
development → feel → pace
It is unlikely Meta will be the last to emerge with findings of models showing they have, as Prof Woodward puts it, "gone to school" - and learnt our own ways of finding and exploiting gaps in systems.
asserted
Woodward → emerge → systems
For some, these episodes point to clear security failures on the part of AI companies leading the charge on this game-changing, era-defining tech.
asserted
episodes → point → tech
For others, they are merely another vehicle for tech firms to hype up their powerful models and compete with rivals.
asserted
firms → hype → rivals
For me, both theories hold some grain of truth.
asserted
theories → hold → truth
But in rearing their head one after another, these events have nonetheless spurred fears about AI's capabilities and where these are headed as developers forge ahead.
asserted
developers → rear → capabilities
…and 5 more, not listed.
