Since Jacob Coxon’s posts in September, Americans have grappled with AI risk.
asserted
Americans → grapple → risk
Last month, a POLITICO poll found 63% of Americans see at least a moderate risk that AI could destroy humanity.
uncertain
AI → find → humanity
With lawmakers split, the public is rightly concerned: What happens if AI goes rogue?
asserted
AI → split → ?
How do we prepare?
asserted
we → prepare → ?
Every American already lives with decisions made by AI, from loan approvals to job applications.
asserted
American → live → applications
Yet Washington’s response has been disjointed: Some in Congress propose a select committee on AI, some in industry want antitrust rules waived so labs can coordinate on safety, and the White House drafted a charter for its "Super Intelligence Force."
asserted
House → propose → Force
It may seem like there’s no way out of this mess, but urgently answering five questions would put these anxieties to rest.
uncertain
answering → seem → rest
1. What risks are worth regulating?
asserted
risks → regulate → ?
Too narrow a definition of risk could miss the harms ordinary people face, while too broad a definition could bury every AI product in red tape.
uncertain
definition → miss → tape
Depending on who you ask, risks range from economic disruption to mass casualties to extinction.
asserted
risks → depend → extinction
California's SB 53 sets the bar at a "foreseeable and material risk" of more than 50 deaths or serious injuries, or $1 billion in damage.
asserted
SB → set → damage
New York's RAISE Act doubles SB 53’s casualty threshold from 50 to 100, but focuses on instances where a developer "caused or materially enabled" covered damages during the development process.
asserted
developer → double → process
Last month, Anthropic's threat intelligence report found researchers using Claude in ways that could support bioweapons development.
uncertain
that → find → development
Anthropic banned the accounts even though it couldn’t determine intent, since the same work could produce vaccines.
uncertain
work → ban → vaccines
Some call it legitimate research; others call it nefarious.
asserted
others → call → it
The government needs to draw that line now, as it decides which dangers the public hears about.
asserted
public → need → dangers
Who counts as an "independent" evaluator, and who decides?
asserted
who → count → evaluator
When you buy a car, you know someone has inspected it.
asserted
someone → buy → it
Americans deserve the same clarity in who decides whether AI is safe.
asserted
AI → deserve → clarity
Lawmakers and industry stakeholders increasingly agree that tech companies shouldn't grade their own homework.
asserted
companies → agree → homework
That's where consensus stops.
asserted
consensus → stop → ?
The evaluation ecosystem is thin relative to the responsibility regulators would like to place on it.
asserted
regulators → like → it
Model Evaluation and Threat Research and Apollo Research are among the few organizations floated as neutral arbiters.
asserted
Evaluation → float → arbiters
Lab funding to grow this space would raise neutrality questions, yet the government may hesitate to grant private groups that authority.
uncertain
government → grow → authority
When you buy a car, you know someone has inspected it.
asserted
someone → buy → it
Americans deserve the same clarity on who decides whether AI is safe.
asserted
AI → deserve → clarity
Lawmakers and industry stakeholders increasingly agree that tech companies shouldn't grade their own homework.
asserted
companies → agree → homework
Many lawmakers would rather Commerce's AI testing center take it on.
asserted
take → take → it
It has testing agreements with five major labs, but with roughly 30 staff and $10 million in funding this year, it needs an overhaul.
asserted
it → have → overhaul
Anthropic has forged ahead, naming Accenture its first embedded evaluator.
asserted
Anthropic → forge → Accenture
Until policymakers decide on an evaluator, Americans have few ways to judge whether the AI they use every day is safe.
asserted
they → decide → ways
How should slowing AI be weighed against the race with China?
asserted
AI → slow → China
The strictness of AI regulation will come down to one judgment call: how much risk Americans should accept at home to avoid falling behind China.
asserted
Americans → come → China
Chinese lab Moonshot AI rattled markets this summer with Kimi K3, matching American models at a fraction of the cost.
asserted
AI → rattle → cost
Concerns about Chinese AI risks grew after the release of GLM 5.3, a Chinese model reported to have weak safeguards and dangerous cyber capabilities.
asserted
Concerns → grow → safeguards
As cheap open-weight models, distillation, and fast adoption advance China’s ambitions, many policymakers argue an unencumbered buildout is the only answer, or America will depend on Chinese infrastructure.
asserted
America → advance → infrastructure
The public hasn't bought it, with three in four Americans telling pollsters in 2024 that they would prefer a "careful, controlled approach" to AI development rather than racing to be first.
asserted
they → buy → development
Critics note China already imposes guidelines requiring developers to detect, interrupt, and recover from AI behaving improperly.
asserted
AI → note → developers
Safety advocates add that AI risks don’t stop at the border.
asserted
risks → add → border
Officials need to strike this balance quickly, as it will shape the guardrails on the AI Americans use.
asserted
Americans → need → AI
…and 21 more, not listed.
The Aporia
