Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation

TL;DR

This paper tackles whether distilled reasoning models truly inherit the teacher's patterns or regress to the student’s baseline in new contexts. It introduces Reasoning Distillation Provenance Tracing, a cross-model framework that compares sentence-level action probabilities $p_{(i,j)}$ among the teacher, student, and distilled models to categorize actions into provenance types such as $Teacher$, $Student$, $Shared$, and $Boosted$. Applying the framework to open-source models on GPQA-D and AIME24 reveals that teacher-originated actions occur early, correlate with correctness, and that distillation also activates latent student patterns, whose usefulness varies by model size. Building on these insights, the authors propose a principled teacher-guided data selection method that favors training examples rich in teacher-originated behavior, demonstrating consistent performance gains across multiple teacher–student pairs and domains. This provenance-aware approach holds promise for more reliable reasoning distillation and targeted data curation in complex reasoning tasks.

Abstract

Reasoning distillation has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student model so that it mimics the teacher's behavior in training contexts. However, previous approaches have lacked a detailed analysis of the origins of the distilled model's capabilities. It remains unclear whether the student can maintain consistent behaviors with the teacher in novel test-time contexts, or whether it regresses to its original output patterns, raising concerns about the generalization of distillation models. To analyse this question, we introduce a cross-model Reasoning Distillation Provenance Tracing framework. For each action (e.g., a sentence) produced by the distilled model, we obtain the predictive probabilities assigned by the teacher, the original student, and the distilled model under the same context. By comparing these probabilities, we classify each action into different categories. By systematically disentangling the provenance of each action, we experimentally demonstrate that, in test-time contexts, the distilled model can indeed generate teacher-originated actions, which correlate with and plausibly explain observed performance on distilled model. Building on this analysis, we further propose a teacher-guided data selection method. Unlike prior approach that rely on heuristics, our method directly compares teacher-student divergences on the training data, providing a principled selection criterion. We validate the effectiveness of our approach across multiple representative teacher models and diverse student models. The results highlight the utility of our provenance-tracing framework and underscore its promise for reasoning distillation. We hope to share Reasoning Distillation Provenance Tracing and our insights into reasoning distillation with the community.

Figures (15)

• Figure 1: Motivation. In reasoning distillation, the student model learns in the training stage to produce actions consistent with the teacher model within the training context. However, at test time, it remains unclear whether the distilled model will continue to output actions aligned with the teacher model instead of degrading to the outputs of the student model, raising concerns about the generalization of distillation models hinton_distilling.
• Figure 2: An illustration of analysis using Reasoning Distillation Provenance Tracing. The horizontal axis denotes the action position (i.e., the sentence order), while the vertical axis shows the probability assigned by different models (indicated by colored curves in the foreground) in producing that action under the same context. The background colors indicate the action types. Some example of different action types are also shown at the top for illustration, where the blank spaces indicate the segmentation boundaries of the response.
• Figure 3: Analysis results on open-source models. We apply Reasoning Distillation Provenance Tracing to Deepseek-Distill-Qwen-7B and DeepSeek-R1-0528-Qwen3-8B, analyzing the probability of producing different actions at each action position on two reasoning benchmarks (AIME24 and GPQA-D). The x-axis denotes the action index, and the y-axis shows the probability of producing that action.
• Figure 4: Analysis results on LIMO-v2 model limov2. We apply Reasoning Distillation Provenance Tracing to LIMO-v2 model, analyzing the probability of producing different actions at each action position on two reasoning benchmarks (AIME24 and GPQA-D). Since LIMO-v2 model uses multiple teacher models, we tried different teacher models on all samples. The x-axis denotes the action index, and the y-axis shows the probability of producing that action.
• Figure 5: Comparison between teacher-guided data selection and the previous method grape.