Once a distant ambition for technology researchers, the prospect of artificial intelligence models teaching themselves autonomously to be more efficient and capable appears ever closer to reality.
asserted
models → teach → reality
As the technology advances, developers say it is approaching “recursive self-improvement,” or RSI, in which AI models find ways to improve themselves and build their successor.
asserted
models → advance → successor
It could bring the promise of advances in science and medicine, tech company executives say, but also risks.
uncertain
executives → bring → science
The uncertainty over where it all could lead is at the heart of growing fears about AI evading human control, and possible threats to humanity, which led several AI moguls to join last weekend in a call to slow down the technology’s pace of growth.
uncertain
which → lead → growth
Anthropic this week detailed how its model Claude is helping the company to develop the next, more intelligent version of itself.
asserted
company → detail → itself
Claude is now leading 26% of Anthropic’s model research and development, which the company said means it can complete most of a given task “end-to-end from a high-level prompt” while still being under human supervision.
asserted
it → lead → supervision
The models are not working completely autonomously — at least not yet.
asserted
models → work → ?
Leading AI companies have different definitions for recursive self-improvement.
asserted
companies → lead → improvement
Some define it as when there is any feedback from AI on model improvement, while others define it as AI working toward that goal fully autonomously.
asserted
AI → define → goal
Autonomous recursive self-improvement essentially means AI that can improve itself designing the next version of the system, then the next version, and so on, said Anthony Aguirre, president and CEO of the nonprofit Future of Life Institute and a physics professor at the University of California, Santa Cruz.
asserted
Aguirre → mean → California
“The really important thing here is that as AI is doing more of it, it gets faster, because AI operates just much, much more quickly than the humans do," he said.
asserted
he → do → it
The fear around RSI is based largely on a runaway superintelligence emerging from that process, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models.
asserted
that → base → models
But he said a more grounded view suggests a kind of recursive self-improvement has been going on in AI development for a while now.
uncertain
view → say → while
"We have already, for years, been using these models in supportive roles for creating the next version of these models.
asserted
We → use → models
So people use the past generation of models to write code for the AI systems that then create the next generation," he said.
asserted
he → use → generation
For years, prominent AI researchers such as OpenAI co-founder Andrej Karpathy have experimented with trying to get AI models to train and improve new AI systems.
asserted
models → experiment → systems
Those efforts have brought minor improvements, but not big creative leaps, Thickstun said.
asserted
Thickstun → bring → improvements
But AI companies today, Aguirre said, are much closer to pulling off those bigger leaps in improvement.
asserted
Aguirre → say → improvement
“You can see in these plots from Anthropic over time, more and more of research is being done by the AI and it’s becoming closer and closer to fully autonomous,” he said.
asserted
he → see → AI
“And the result of that success, ultimately is something that is, I think, extremely scary.
asserted
I → think → success
I think this is probably the worst idea in the history of humanity to do this.
asserted
this → think → this
And yes, they’re doing it.
asserted
they → do → it
Some AI labs say RSI is not far off
Anthropic's recent announcement provided the public — and other labs — with some insight into RSI progress, and it encouraged its competitors to share similar metrics.
asserted
it → say → metrics
Still, the company has not expressly said how close it is to achieving fully autonomous model improvement.
asserted
it → say → improvement
ChatGPT maker OpenAI announced this month that it has developed an automated “research intern,” which it defines as a system that can carry out well-defined research tasks under human direction, including “tasks that would take a skilled researcher a few days.”
asserted
that → announce → days
The company has said it is moving forward with the goal of creating an automated AI “researcher” by March 2028.
asserted
it → say → March
The company said in that announcement that while RSI can help align models' actions with human values and intentions, that doesn't mean “rapid RSI is necessarily an outcome we should pursue.”
asserted
we → say → values
“Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks,” the company said in a blog post.
asserted
company → proceed → post
Elon Musk seems more eager to forge ahead.
asserted
Musk → seem → ?
He said in March that for xAI's Grok models, “humans are gradually getting less and less in the loop” on model improvement and that “every successive model is built by the one before it,” but clarified that the process was not yet fully automated.
asserted
process → say → it
That target might be reached by the end of this year, he added, “but not later" than 2027.
uncertain
he → reach → 2027
Microsoft and some other leading AI companies seem to be taking a different approach.
asserted
Microsoft → lead → approach
Mustafa Suleyman, the CEO of Microsoft AI, has said the company is moving toward “humanist superintelligence,” or advanced AI capabilities that are in service of people and humanity at large.
asserted
that → say → people
Suleyman said in a 2025 essay that this would not mean “an unbounded and unlimited entity with high degrees of autonomy,” but rather AI that is “carefully calibrated, contextualized, within limits.”
asserted
that → say → limits
How development slowdown talks could impact RSI
A key challenge labs face — and have been facing essentially since the technology’s inception — is ensuring their safety measures advance alongside the models' capabilities.
uncertain
measures → impact → capabilities
Divisions have emerged in the tech industry over calls for a coordinated AI slowdown for safety, and not every major player in the AI space has specifically commented on their path forward with RSI.
asserted
player → emerge → RSI
Anthropic, which has been a leading voice in the calls for pacing, has said it would slow or temporarily pause its development work — assuming its global competitors also did so, and in a "verifiable manner.”
asserted
competitors → lead → manner
OpenAI explicitly said this month it does not yet know how to “safely get all the way to aligned, full RSI,” adding that the company “cannot assume that progress in alignment and safety will keep pace.”
asserted
progress → say → pace
More capable systems can become harder to monitor, it continued, but pursuing RSI is still a goal it says it values because an “automated AI researcher can also be an automated safety or alignment researcher.”
asserted
researcher → become → RSI
The Aporia
