For months, Anthropic and OpenAI have warned that AI may be on the verge of improving itself.
uncertain
AI → warn → itself
This tipping point, known as “recursive self-improvement,” or RSI, could mean that new AI capabilities arrive faster than ever.
uncertain
capabilities → know → improvement
But the advent of RSI might also be highly perilous for humanity, potentially marking the stage at which developers lose control of AI systems entirely.
uncertain
developers → mark → systems
Even as AI companies warn about the potential dangers of RSI, they have shared remarkably little about how they’re pursuing it.
asserted
they → warn → it
A new report, published Wednesday by the firm Gladstone AI, attempts to change that.
asserted
report → publish → that
It gives one of the most detailed pictures yet of how RSI could work and how AI companies are progressing toward it.
uncertain
companies → give → it
The report is based on what the authors say are conversations with half a dozen researchers inside the top AI companies.
asserted
authors → base → companies
The report, exclusively shared with TIME ahead of its publication, emphasizes that AI companies are “actively working on” automating all parts of the AI development pipeline, including generating new ideas, running experiments to test those ideas, and interpreting the results.
asserted
companies → share → results
Most of the insiders Gladstone spoke to “believe RSI will start within the next 3 to 12 months, and possibly sooner, assuming there’s no U.S. government intervention or slowdown,” the report states.
uncertain
report → speak → months
“There's a fair amount of [disaster] prepping and bio-bunker purchasing going on” among these researchers, says Edouard Harris, Gladstone’s chief technology officer.
asserted
Harris → be → researchers
The aim of the report, Gladstone’s executives tell TIME, is to give lawmakers a better understanding of how RSI works, and, in turn, a greater ability to regulate it.
asserted
RSI → tell → it
“Our goal here was to define recursive self-improvement well enough that people can have a sane conversation about it, especially in Washington,” says Jeremie Harris, Gladstone’s CEO.
asserted
Harris → define → Washington
“There are all these bills proposed to ban superintelligence, which are really interesting attempts to grapple with this problem.
asserted
which → be → problem
But what we're more interested in is: what is the process that needs to be regulated?”
asserted
that → need → what
How RSI could work
The report gives a simplified outline of how the AI research pipeline currently operates inside AI companies—and how those companies are beginning to automate the process.
uncertain
companies → work → process
For the first phase of training an AI model, known as pretraining, much of the emphasis is on finding ways to squeeze more intelligence out of a model with less and less computing power, known in the industry as “compute.”
asserted
much → train → compute
The AI companies take a portfolio approach to testing these ideas.
asserted
companies → take → ideas
Researchers allocate a small amount of compute to each idea, using it to train a series of models.
asserted
Researchers → allocate → models
The models that reliably perform better as they scale up are selected for another round of training, using larger amounts of compute.
asserted
they → perform → compute
Companies don’t have infinite compute, which limits the number of promising ideas they can test at larger scales.
asserted
they → have → scales
This means that even if AI companies can train an AI that is able to generate ideas that seem promising, it matters greatly how many bad ideas it generates simultaneously.
asserted
it → mean → ideas
If companies need to spend huge amounts of time or compute filtering good AI-generated ideas from bad ones, it might make sense to continue to rely on human researchers with sound intuition.
uncertain
it → need → intuition
Conversely, AI companies with access to more compute are able to test more ideas—potentially allowing them to reach RSI first.
asserted
them → test → RSI
With more compute to throw at experiments, companies may also be likelier to catch ideas that the Gladstone report describes as the rarest and most desired by AI companies: those that perform better as more compute is thrown at them, even if they appear to be duds at smaller scales.
uncertain
they → throw → scales
Being able to tell when those ideas are found—and particularly whether AI is helping find them—“is just such a critical part of this process that right now legislators have essentially zero visibility into,” says Jeremie Harris.
asserted
Harris → tell → visibility
One of the traits that makes regulating RSI tricky is that it is a gradual process.
asserted
it → make → RSI
“This is actually one of the reasons why we went to pains to describe it in so much detail,” says Edouard Harris.
asserted
Harris → go → detail
“So that it is possible to point to a part of this process and say: a human needs to be there.”
asserted
human → point → process
The Aporia
