Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces

Shaltiel Shmidman, Asher Fredman, Oleg Sudakov, Meriem Bendris

TL;DR

The paper addresses how test-time computation through reasoning traces can boost medium-sized LLMs for math problem solving. It compares two open-source reasoning styles, DeepSeek-R1 and gpt-oss, by fine-tuning two 12B base models on a large math dataset (242k samples) that pairs problems with full traces. Results show similar downstream accuracy across benchmarks, but gpt-oss traces are approximately four times shorter, offering substantial inference efficiency gains. The findings highlight that verbosity does not strictly improve performance and motivate future work on hybrid training and domain generalization.

Abstract

Test-time scaling, which leverages additional computation during inference to improve model accuracy, has enabled a new class of Large Language Models (LLMs) that are able to reason through complex problems by understanding the goal, turning this goal into a plan, working through intermediate steps, and checking their own work before answering . Frontier large language models with reasoning capabilities, such as DeepSeek-R1 and OpenAI's gpt-oss, follow the same procedure when solving complex problems by generating intermediate reasoning traces before giving the final answer. Today, these models are being increasingly used to generate reasoning traces that serve as high-quality supervised data for post-training of small and medium-sized language models to teach reasoning capabilities without requiring expensive human curation. In this work, we compare the performance of medium-sized LLMs on Math problems after post-training on two kinds of reasoning traces. We compare the impact of reasoning traces generated by DeepSeek-R1 and gpt-oss LLMs in terms of accuracy and inference efficiency.

Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces

TL;DR

The paper addresses how test-time computation through reasoning traces can boost medium-sized LLMs for math problem solving. It compares two open-source reasoning styles, DeepSeek-R1 and gpt-oss, by fine-tuning two 12B base models on a large math dataset (242k samples) that pairs problems with full traces. Results show similar downstream accuracy across benchmarks, but gpt-oss traces are approximately four times shorter, offering substantial inference efficiency gains. The findings highlight that verbosity does not strictly improve performance and motivate future work on hybrid training and domain generalization.

Abstract

Test-time scaling, which leverages additional computation during inference to improve model accuracy, has enabled a new class of Large Language Models (LLMs) that are able to reason through complex problems by understanding the goal, turning this goal into a plan, working through intermediate steps, and checking their own work before answering . Frontier large language models with reasoning capabilities, such as DeepSeek-R1 and OpenAI's gpt-oss, follow the same procedure when solving complex problems by generating intermediate reasoning traces before giving the final answer. Today, these models are being increasingly used to generate reasoning traces that serve as high-quality supervised data for post-training of small and medium-sized language models to teach reasoning capabilities without requiring expensive human curation. In this work, we compare the performance of medium-sized LLMs on Math problems after post-training on two kinds of reasoning traces. We compare the impact of reasoning traces generated by DeepSeek-R1 and gpt-oss LLMs in terms of accuracy and inference efficiency.

Paper Structure

This paper contains 11 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Math Reasoning Dataset
  3. Experimental Setup
  4. Base Models
  5. Infrastructure
  6. Training Details
  7. Results
  8. Discussion
  9. Conclusion
  10. Acknowledgments
  11. Appendix: System Prompt used for verifying correctly generated answer

Figures (1)

  • Figure 1: Graph comparing the training loss when fine-tuning Nemotron-Nano-12B-V2 on the two datasets. The DeekSeek-R1 dataset doesn't seem to significantly affect the loss, which is most likely because of the introduction of the reasoning traces in the mid-training.