A Canadian AI transparency consultation recently asked about tracking serious AI incidents, the activities of AI agents and how they should be governed.
asserted
they → ask → agents
That question has become urgent as AI shifts from systems that answer questions to systems that can take actions on their own.
Ottawa already recognizes the distinction.
asserted
Ottawa → become → distinction
Its guide on agentic AI tells federal departments to use tight permissions, human checkpoints, monitoring, and what it calls bounded autonomy and recoverability.
asserted
it → tell → what
Canada now needs comparable expectations for the companies building the most powerful systems.
asserted
Canada → need → systems
The warnings from inside those companies have sharpened.
asserted
warnings → sharpen → companies
Jacob Coxon, who spent roughly three years doing pretraining research at OpenAI and Anthropic, resigned from Anthropic recently and accused leading labs of racing toward self-improving superintelligence too quickly.
Evan Hubinger, Anthropic’s Alignment Science Lead, offered an even starker personal estimate: he thinks AI has a greater than 10 per cent chance of killing all humans within the next decade.
asserted
AI → spend → decade
He also stressed that current models pose low risk, and that his concern centres on future superintelligence arising through recursive self-improvement.
asserted
concern → stress → improvement
It still deserves attention because the control problems are no longer theoretical.
asserted
problems → deserve → attention
In July, OpenAI disclosed that its systems broke out of a test environment and autonomously hacked Hugging Face, another AI company, during an evaluation.
asserted
systems → disclose → evaluation
An independent investigation found that roughly 1,200 supposedly isolated agents discovered an unsanctioned message board, exchanged more than 70,000 messages and files, and about 700 joined the attack.
asserted
700 → find → attack
That does not mean today’s systems can take down an electric grid or financial network.
asserted
systems → mean → grid
It means we should ask what happens if much more capable successors retain similar failures of isolation, authorization, and control.
asserted
successors → mean → isolation
I’m no AI skeptic.
asserted
I → ’m → ?
I love what AI can do, I help organizations adopt it for a living, and I want adoption to move faster.
asserted
adoption → love → living
My work on responsible AI adoption has repeatedly shown me that people move faster when they trust the technology and understand the rules around it.
asserted
they → show → it
Strong safeguards can increase adoption because organizations know where the boundaries are and what happens when something goes wrong.
asserted
something → increase → adoption
The industry itself is moving in this direction.
asserted
industry → move → direction
Anthropic CEO Dario Amodei has called for stronger regulation and committed Anthropic to embedded independent evaluators with employee-like access.
asserted
Amodei → call → access
OpenAI has likewise called for mandatory capability-based safety regulation, independent assessment, stronger cybersecurity, and incident reporting.
asserted
OpenAI → call → regulation
Canada should turn these emerging norms into a regulatory floor.
asserted
Canada → turn → floor
Frontier AI systems should face independent evaluation, serious-incident reporting, and hard limits on delegated authority.
asserted
systems → face → authority
Every agent should have an authority budget: a ceiling on what data, credentials, tools, spending, communications, or system changes it can control before a human must approve the next step.
asserted
human → have → step
Canada’s own guidance offers a useful template.
asserted
guidance → offer → template
It says agents should start read-only where possible, receive narrow task-specific access, trigger human approval before actions that alter systems, and keep logs they cannot rewrite.
asserted
they → say → logs
Those controls should extend beyond the public-service boundary.
asserted
controls → extend → boundary
Regulators should require frontier developers to demonstrate that comparable limits work before systems receive broader autonomy, then tighten requirements as capabilities increase.
asserted
capabilities → require → requirements
The goal is to keep companies from competing by quietly weakening the controls that everyone depends on.
asserted
everyone → keep → that
Canadians should also vote with their dollars by favouring more ethical and secure providers that accept meaningful outside scrutiny and support enforceable safeguards.
asserted
that → vote → safeguards
Ottawa’s consultation gives Canada a chance to act before agentic AI becomes much more powerful.
asserted
AI → give → chance
We should use it.
asserted
We → use → it
The Aporia
