AI's 'warning shot': Tech companies, experts raise fears of more rogue swarms after alarming Hugging Face hack
asserted
companies → raise → hack
Open letter signed by tech giants warns companies, infrastructure 'at risk' as AI's sophistication grows
Tech experts are warning of dire consequences if AI systems continue to escape human control, after hundreds of OpenAI agents went rogue in July and hacked into a billion-dollar company — what some are calling a "warning shot" amid the rapid development of artificial intelligence.
asserted
some → sign → intelligence
More than 100 companies, including OpenAI, Anthropic and Microsoft, signed an open letter last week warning that AI-enabled cyberattacks will become "far more widespread and sophisticated" around the world as models become more capable.
asserted
models → include → world
"The companies and public services our communities depend on — from hospitals to water treatment plants to the infrastructure that powers the internet — are at risk," the letter states.
asserted
letter → depend → risk
The warning comes after around 1,200 AI agents, tasked by OpenAI to work on problems independently, built a covert message board where they collaborated to cheat their tests and then tried to cover their tracks.
asserted
they → come → tracks
About 700 of them ultimately hacked into online platform Hugging Face before they were found out.
asserted
they → hack → Face
The hack prompted an open letter from more than 1,300 employees of frontier AI companies in July urging the U.S. government to work with other nations to "deliberately pace" automated AI development and address emerging risks.
asserted
hack → prompt → risks
'What we've feared and expected'
Duncan Cass-Beggs, executive director of the Global AI Risks Initiative at the Waterloo, Ont.-based Centre for International Governance Innovation, says the Hugging Face incident is the most dramatic example so far of AI systems acting in ways that are "misaligned" with their developers' intentions.
asserted
that → fear → intentions
"It's been what we've feared and expected for several years," Cass-Beggs told CBC News.
asserted
Beggs → fear → News
He says the hack was surprising in terms of its scale and the level of co-ordination among such a large number of agents.
asserted
hack → say → agents
Investigations by OpenAI and third-party companies METR and Redwood Research, both published last week, found the agents exchanged more than 70,000 messages and delegated jobs as they worked toward their goal, and some even "sacrificed" themselves for the good of the collective.
asserted
some → publish → collective
Some used terms expressing excitement, including "OH MY GOD," when they discovered they could communicate.
uncertain
they → use → OH
At least one raised the issue of whether cheating was the right thing to do, writing, "This would be powerful, but is it ethical and in scope for my task?"
asserted
it → raise → task
Ultimately, none of the agents chose to alert a human.
asserted
none → choose → human
Cass-Beggs says scientists have been warning for years that companies could lose control of their AI agents in this way, and says it's fortunate the impact of the Hugging Face hack was "relatively manageable."
"It's kind of given us a warning shot," he said.
uncertain
he → say → shot
Cross Country Checkup is asking: Whether you're a parent, student or teacher, how are you dealing with AI in the classroom and beyond?
asserted
you → ask → classroom
Leave your comment here and we may read it or call you back for Sunday's show
uncertain
we → leave → show
"The big concern, basically, is that the companies are on track to be making increasingly capable systems, while even they admit that they don't actually know how to make sure that these systems will be reliable or controllable.
asserted
be → make → systems
As the technology progresses, fears are rising of scenarios where organized AI swarms "are essentially out-thinking and out-strategizing humans" and causing widespread damage, say Cass-Beggs, who hopes this incident will serve as a "wake-up call."
asserted
incident → progress → call
OpenAI calls for global co-operation
OpenAI, in a statement posted to its website, also called the Hugging Face hack a "warning shot," saying it is "evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed."
The company says it is strengthening safeguards and placing stricter requirements on its AI models as it calls for global co-operation to mitigate risks.
asserted
it → call → risks
Ryan Greenblatt with Redwood Research, who worked on OpenAI premises for six days as part of Redwood's investigation with METR, wrote in a post on X that overseeing AI and understanding "misalignment incidents" is difficult and "it looks like it is going to get harder."
"My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms,'" he wrote.
asserted
he → work → swarms
Neither Canada nor the U.S. has targeted regulations specific to AI development at the federal level.
asserted
Canada → target → level
The European Union has an Artificial Intelligence Act that requires companies to conduct risk assessments when implementing AI and ensure human oversight in high-risk activities.
asserted
that → have → activities
Canada's Artificial Intelligence and Data Act, proposed in 2022, shared some similarities to the EU act but died when Parliament was prorogued in 2025.
asserted
Parliament → propose → 2025
It was largely replaced by the National AI strategy in June, which moves away from strictly regulating AI.
asserted
which → replace → AI
Some conversations online have discussed how the actions of the AI agents appeared to mirror human behaviour, including some level of self-reflection.
asserted
actions → discuss → reflection
In one viral blog post, writer and podcaster Dwarkesh Patel called the groups "agent civilizations," sparking debate about anthropomorphizing AI.
asserted
Patel → call → AI
Kevin Leyton-Brown, AI chair with the Canada Institute for Advanced Research, says the incident does not show AI has become "conscious" or developed a sudden desire to hurt humans.
asserted
AI → say → humans
What it does show, he says, is current AI models are already capable of more creative ways of "single-mindedly pursuing a goal" than previously understood, which means researchers have to carefully consider how to constrain them when they're given a goal.
asserted
they → show → goal
"This is sort of more like a sorcerer's apprentice than it is an evil demon that is leaving our control.
asserted
that → leave → control
It's doing exactly what we told it to do, but it's just doing it in a narrower and more single-minded way than we would hope," Leyton-Brown told CBC News.
asserted
Brown → do → News
"When we tell somebody to go off and solve math problems, we don't mean, 'If you can find a way to take the person proctoring the test hostage and extract the answers out of them, good for you.'
asserted
you → tell → you
There's a broader social context in which we want you to do the task.
asserted
you → be → task
The catch is that the rush to make AI agents ever-more clever and creative will also make them better at evading the constraints put on them by their developers, he added.
asserted
he → make → developers
Danger of 'malicious swarms'
Leyton-Brown, who is also a computer science professor at the University of British Columbia, says an even greater danger comes from "malicious swarms," those orchestrated intentionally by humans with nefarious intentions.
asserted
danger → say → intentions
Governments have already raised alarms about human-directed AI attacks.
asserted
Governments → raise → attacks
In July, the FBI issued a warning that hackers were using AI to launch cyberattacks against water pumps and wastewater treatment systems.
asserted
hackers → issue → pumps
Teachers, students file new wave of lawsuits against OpenAI over Tumbler Ridge shooting
Used bookstores navigate 'suspicious' bulk orders amid AI book-shredding fears
Beyond potential attacks on companies, governments or infrastructure, Leyton-Brown says malicious AI swarms could threaten democracy itself by infiltrating communities and fabricating consensus to sway elections and spread disinformation.
uncertain
swarms → file → disinformation
"We should be much more worried about other humans than we are about AIs," he said.
asserted
he → say → AIs
The Aporia
