arxiv:2609.14858
Published on Sep 14
· Submitted by
TongZheng on Sep 15
Upvote
242
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Dream-RSI enables scalable recursive self-improvement by using historical discovery replay to evaluate exploration policies offline, reducing costly online evaluations.
Generated by thinkingmachines/Inkling-Small
Recursive self-improvement is becoming increasingly vital for autonomous AI agents, where progress hinges on discovering high-value solutions across complex domains. The driver of this process is effective exploration, however, managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization requires navigating vast meta-search spaces under delayed and expensive feedback over long-horizon rollouts. We introduce Dream-RSI, a framework for scalable and recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying coding agent unchanged. Our key insight is that accumulated discovery history can serve as a replay simulator over the realized search space. By performing dreaming in the replay simulator constructed from historical discovery trees, Dream-RSI secures immediate, low-cost off-policy feedback to evaluate and refine exploration policies without invoking repetitive, expensive online evaluations. The improved policy is subsequently redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings.
View arXiv page View PDF Project pageGitHub 1.06k Add to collection
Community
Paper submitter 8 days ago
To recursively self-improve, agents need to dream. History can be the world they dream in.
More Details: https://dream-rsi.com/
7 days ago
This is an automated message from the ResearchStudio team.
We created an interactive ResearchStudio Reel for this paper. It includes a visual poster, a video, and a blog, all available for download in editable formats.
Open the ResearchStudio Reel →
Download all files from HF Mirror
Please give this comment a thumbs up if you find the Reel helpful!
Want to explore or create Reels for more papers? Visit the ResearchStudio demo.
7 days ago
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
-
ADIAS: Automated Design of Interactive Agentic Systems (2026)
-
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses (2026)
-
Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineering (2026)
-
AREX: Towards a Recursively Self-Improving Agent for Deep Research (2026)
-
Experience Funnel: A State-Policy Alternating Loop for Self-Evolving Agents (2026)
-
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution (2026)
-
Procedural Graphs: Self-Evolving Execution Structures for LLM Agents (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on HF Mirror checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
· Sign up or log in to comment
Upvote
242
Get this paper in your agent:
hf papers read 2609.14858
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2609.14858 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2609.14858 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2609.14858 in a Space README.md to link it from this page.