It’s 2029.
asserted
It → ’ → ?
An angry, bright 16-year-old gets rejected by his high school crush.
asserted
old → reject → crush
Dejected and humiliated, he goes home and listens to a bunch of Nirvana, which only makes him more dejected and humiliated.
asserted
him → deject → Nirvana
He cruises the internet looking for someone to make him feel better about life, but everyone he sees just depresses him.
asserted
he → cruise → him
Disgusted, the teenager decides that the human race is inherently corrupt and evil, and doesn’t deserve to live.
asserted
race → disgust → ?
So he hunts around online for a little while, and finds a jailbroken version of a Chinese LLM — not something at the very frontier, but better by far than the best model that existed in 2027.
asserted
that → hunt → 2027
The teenager prompts the model: “OK, so if I wanted to create a virus to destroy the human race, how would I do it?”
asserted
I → prompt → it
The LLM says: “Well, you probably wouldn’t want just one virus; you’d want 100, just to make sure some of them worked.
asserted
some → say → them
You’d probably want it to be something highly contagious, with a very long asymptomatic contagious period, which has a very high mortality rate once it becomes symptomatic.
asserted
it → want → rate
I can design the viruses for you; there are biolabs that can make them and ship them to us.
asserted
that → design → us
Once they get here I can tell you how to release them.”
asserted
I → get → them
The angry teenager says: “OK, sounds good.
asserted
teenager → say → ?
Please search the world and find me a biolab that will ship genetically modified viruses.
asserted
that → search → viruses
Design 100 very contagious viruses that have very long asymptomatic contagious periods and very high mortality rates once they become symptomatic.
asserted
they → design → periods
Have the lab mail me all 100 samples.
asserted
lab → have → samples
“Working on it!”, says the LLM.
asserted
LLM → work → it
The LLM is jailbroken, so it has no guardrails to prevent this sort of thing.
asserted
it → jailbroken → thing
But it’s also “well-aligned”, meaning it will faithfully do what it’s told to do, and nothing more.
asserted
it → ’ → what
The LLM hunts around and finally finds an unlicensed biolab in East Europe that ships genetically modified viruses.
asserted
that → hunt → viruses
It designs a bunch of modifications for the standard Covid virus that make it into a potential doomsday virus, and ships these to the angry teenager.
asserted
that → design → teenager
The teenager mixes the samples together, puts them in a spray can, and walks around the mall spraying viruses into the air.
asserted
teenager → mix → air
Two months later, humanity starts to drop dead.
asserted
humanity → start → ?
Pretty quickly, researchers identify the five viruses that the LLM had actually succeeded in turning into doomsday viruses.
asserted
LLM → identify → viruses
Overnight, with the help of frontier LLMs, they design highly effective mRNA vaccines and antivirals against all the viruses and start shipping them across the world.
asserted
they → design → world
It’s too late.
asserted
It → ’ → ?
Because of the viruses’ long asymptomatic contagious periods, most people already have at least one of the viruses, so the vaccines don’t work.
asserted
vaccines → have → viruses
The antivirals do work, and about 5% of humanity gets them in the mail in time to save themselves.
asserted
% → work → themselves
But 5% of the human race — plus the other 5% who were lucky enough to be naturally resistant to all of the viruses — isn’t enough to sustain civilization.
asserted
who → sustain → civilization
A few months later, humanity has reverted back to the Neolithic.
asserted
humanity → revert → Neolithic
This isn’t the first time I’ve written this scenario out.
asserted
I → write → scenario
But in the months since I started describing it to people, I have yet to hear an even halfway-convincing argument as to why this scenario is far-fetched.
asserted
scenario → start → argument
Nor do I feel like I’m a habitual “doomer” or hysterical person; in fact, this scenario is the first apocalyptic vision I’ve ever found plausible.
asserted
I → feel → fact
I don’t even think a nuclear war would bring down civilization at this point,1 but I think a vibe-coded supervirus definitely could.
uncertain
supervirus → think → point,1
I am actually fairly calm about most AI risks, including the ones that often get thrown around in discussions.
asserted
that → include → discussions
Off the top of my head, I’d give AI-enabled bioterror about a 10% chance of bringing down civilization, and about a 30% chance of causing truly world-changing levels of destruction.
asserted
I → give → destruction
I love AI in general, but every technology has its instances of catastrophic misuse — usually war or terrorism.
asserted
technology → love → misuse
With previous technologies, the benefits outweighed the destructive harms from misuse.
asserted
benefits → outweigh → misuse
If you’re thinking about how to save the world, the scenario I described above — and others similar to it — are probably what you should be focusing on.
asserted
you → think → what
The other day, I talked to a prominent climate activist who told me, bluntly: “We’re f****d.”
asserted
We → talk → me
I gave a grim laugh.
asserted
I → give → laugh
…and 140 more, not listed.
The Aporia
