NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks

TL;DR

NaturalThoughts investigates how to best distill general reasoning from a strong teacher to smaller models by curating reasoning traces across a large, diverse prompt pool. The authors show that scaling data size with random sampling yields reliable gains, but selecting difficult examples with diverse reasoning strategies improves sample efficiency the most. They introduce a mixed System-1/System-2 distillation approach that enables flexible inference-time tradeoffs between accuracy and efficiency, achieving strong results on broad STEM benchmarks (GPQA-Diamond, MMLU-Pro, SuperGPQA) with Llama and Qwen student models. The findings emphasize data diversity and difficulty over topic diversity for learning robust reasoning, offering practical guidance for building compact reasoning models. The work also demonstrates the value of adaptive inference modes and systematic data-driven ablations to understand what finally enables general reasoning in smaller models.

Abstract

Recent work has shown that distilling reasoning traces from a larger teacher model via supervised finetuning outperforms reinforcement learning with the smaller student model alone (Guo et al. 2025). However, there has not been a systematic study of what kind of reasoning demonstrations from the teacher are most effective in improving the student model's reasoning capabilities. In this work we curate high-quality "NaturalThoughts" by selecting reasoning traces from a strong teacher model based on a large pool of questions from NaturalReasoning (Yuan et al. 2025). We first conduct a systematic analysis of factors that affect distilling reasoning capabilities, in terms of sample efficiency and scalability for general reasoning tasks. We observe that simply scaling up data size with random sampling is a strong baseline with steady performance gains. Further, we find that selecting difficult examples that require more diverse reasoning strategies is more sample-efficient to transfer the teacher model's reasoning skills. Evaluated on both Llama and Qwen models, training with NaturalThoughts outperforms existing reasoning datasets such as OpenThoughts, LIMO, etc. on general STEM reasoning benchmarks including GPQA-Diamond, MMLU-Pro and SuperGPQA.

Figures (11)

• Figure 1: Comparison of NaturalThoughts (NT) with existing distillation datasets, when training Llama-3.1-8B-Instruct (Top) and Qwen-2.5-7B-Instruct (Bottom) respectively. In contrast to the "Less is More" hypothesis ye2025limoreasoningmuennighoff2025s1simpletesttimescaling, we observe that scaling up high-quality questions and reasoning demonstrations consistently improves performance, even with random selection. Selection based on diversity in reasoning strategies and difficulty (e.g. long CoT, disagreement between teacher models) further improves over random selection (details in \ref{['subsec:selection']} and \ref{['sec:data_selection']}).
• Figure 2: A mix of System-1 (no reasoning traces, final answer only) and System-2 (full reasoning traces) distillation improves the inference-time efficiency of the student model (details in \ref{['subsec:mix_reasoning']} and \ref{['sec:distillation_methods']}).
• Figure 3: Top 20 commonly used reasoning strategies based on annotations of 10,000 samples. We observe that there are a small number frequently used strategies, followed by a long tail of more niche strategies.
• Figure 4: NaturalThoughts Overview. We outline the three main components of our strategies for selecting and distilling reasoning data. 1) Starting from the NaturalReasoning dataset yuan2025naturalreasoning, we sample prompts for the teacher model to generate reasoning and answer traces from, creating the NaturalThoughts dataset. 2) From the NaturalThoughts seed set, we use several filtering methods for data selection (\ref{['sec:data_selection']}). 3) Given a filtered subset obtained from data selection, we finetune a student model on the prompt-reasoning-answer triples, where the reasoning traces and answers are generated by the teacher model.
• Figure 5: Generation length distributions of System-1 and System-2 reasoning for GPQA-Diamond. System-1 responses (left) are predominantly brief, indicating minimal thinking, if any. In contrast, System-2 responses (right) are significantly longer than those of System-1 and exhibit greater variance, as the response lengths vary based on the amount of thinking required for each question, depending on its complexity.