Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning

TL;DR

This work revisits the dominant practice of fine-tuning small models on teacher-generated data (sequence-level distillation) for long-chain-of-thought reasoning and identifies key gaps in representing the teacher’s full output distribution, aligning it with the student’s learning, and mitigating exposure bias. The authors introduce Distribution-Aligned Sequence Distillation (DASD), a data-efficient pipeline built on three pillars: temperature-scheduled learning to broaden mode coverage, divergence-aware sampling to target teacher-originated content aligned with the student, and mixed-policy distillation to reduce training-time exposure bias. Demonstrated on a 4B student and a 120B teacher, the approach achieves state-of-the-art results across mathematics, coding, and scientific reasoning using only 448K distilled samples, and extends to MoE-scale variants with competitive performance. The work provides an open-source dataset and models, highlighting a practical path to high-performing, compact, open reasoning systems and suggesting avenues for refinement via distribution-aware reweighting and agentic capabilities.

Abstract

In this report, we introduce DASD-4B-Thinking, a lightweight yet highly capable, fully open-source reasoning model. It achieves SOTA performance among open-source models of comparable scale across challenging benchmarks in mathematics, scientific reasoning, and code generation -- even outperforming several larger models. We begin by critically reexamining a widely adopted distillation paradigm in the community: SFT on teacher-generated responses, also known as sequence-level distillation. Although a series of recent works following this scheme have demonstrated remarkable efficiency and strong empirical performance, they are primarily grounded in the SFT perspective. Consequently, these approaches focus predominantly on designing heuristic rules for SFT data filtering, while largely overlooking the core principle of distillation itself -- enabling the student model to learn the teacher's full output distribution so as to inherit its generalization capability. Specifically, we identify three critical limitations in current practice: i) Inadequate representation of the teacher's sequence-level distribution; ii) Misalignment between the teacher's output distribution and the student's learning capacity; and iii) Exposure bias arising from teacher-forced training versus autoregressive inference. In summary, these shortcomings reflect a systemic absence of explicit teacher-student interaction throughout the distillation process, leaving the essence of distillation underexploited. To address these issues, we propose several methodological innovations that collectively form an enhanced sequence-level distillation training pipeline. Remarkably, DASD-4B-Thinking obtains competitive results using only 448K training samples -- an order of magnitude fewer than those employed by most existing open-source efforts. To support community research, we publicly release our models and the training dataset.

Figures (10)

• Figure 1: Performance of DASD-4B-Thinking on benchmark datasets. All metrics for the comparison models are taken from their official reports.
• Figure 2: Overall training pipeline of DASD-4B-Thinking.
• Figure 3: Comparison of probability distribution and training loss with data sampled from gpt-oss-120b under different temperatures. We randomly sampled 50K mathematical reasoning responses at both low (T=0.6) and high (T=1.0) temperatures. To characterize the overall likelihood of a response, we compute the geometric mean of its token-level probabilities. (a) Probability distributions of sampled responses: the upper panel displays the density of probability distribution, while the lower panel shows the probability intervals covered by the sampled responses. (b) SFT training loss curves for the student model trained on responses sampled at these temperatures.
• Figure 4: Comparison of probability distribution and training loss with data sampled from Qwen3-Next-80B-A3B-Thinking under different temperatures.
• Figure 5: Joint comparison of the three models’ predicted probabilities. An example of output probabilities: the x-axis indexes sentences, and the y-axis shows predicted probabilities. Foreground lines plot the probabilities of the three models, while background colors indicate the inferred source of each sentence. By comparing probability differences, every sentence is categorized into one of four source types.