arxiv:2609.20612
Published on Sep 17
· Submitted by
XiuYu Zhang on Sep 18
· National University of Singapore
Upvote
35
Authors:
,
,
,
,
Abstract
On-policy self-distillation (OPSD) lets a language model learn from a frozen copy of itself that sees an answer or a worked solution. Giving the teacher this extra information seems to offer the student more to learn, but how much does it add beyond distillation itself? To isolate that contribution, we construct AMPLE-Math, a reusable suite of 5,319 mathematical problems with six reasoning views that share the same answer, and compare each view with matched reference-free distillation. With a thinking-enabled teacher supervising direct-response rollouts, reference-free distillation accounts for much of Qwen3-1.7B's improvement under thinking-enabled evaluation, both in domain and on external benchmarks. Evidence for an additional reference benefit is modest in Qwen, strongest for a polished solution, whereas complete traces add two percentage points in SmolLM3-3B at step 50. These benefits depend on the student being trained. At the same checkpoint, replacing short direct-response rollouts with long thinking-enabled rollouts turns gains into losses in both families while the problems, references, and evaluation stay fixed. Teacher profiles and matched loss interventions in Qwen further show that changing token-level supervision can leave student behavior largely unchanged. Together, these findings suggest that OPSD can improve access to existing reasoning capabilities through parameters shared by direct-response and thinking-enabled inference. The value of a privileged reference is what it adds to this cross-mode transfer, not how much of the solution it reveals.
View arXiv page View PDF GitHub 5 Add to collection
Community
Paper submitter 5 days ago
Giving the teacher more of the solution does not necessarily give the student more to learn. We introduce AMPLE-Math (https://hf-mirror.com/datasets/xiuyuz/ample-math) to isolate what privileged references add beyond the existing asymmetry in the OPSD training recipe, showing across two language-model families that their value depends on how the student is trained and evaluated.
4 days ago
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
-
Rethinking Privileged Information in On-Policy Self-Distillation (2026)
-
DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation (2026)
-
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation (2026)
-
VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation (2026)
-
PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation (2026)
-
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on HF Mirror checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
· Sign up or log in to comment
Upvote
35
Get this paper in your agent:
hf papers read 2609.20612
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2609.20612 in a model README.md to link it from this page.
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2609.20612 in a Space README.md to link it from this page.