Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning

Kehua Feng, Keyan Ding, Zhihui Zhu, Lei Liang, Qiang Zhang, Huajun Chen

TL;DR

CoT-Evo introduces an evolutionary framework for distilling chain-of-thought reasoning tailored to scientific tasks. It builds a diverse candidate pool from multiple thinkers, augments knowledge, and refines CoTs through novelty-driven selection, reflective recombination, and mutation within an evaluation loop. A fitness function combining correctness, length appropriateness, and knowledge usage guides evolution, producing a compact, high-fidelity CoT dataset for training smaller models. Empirical results on BioProBench and ChemCoTBench show consistent improvements over traditional distillation baselines and competitive performance against larger LLM teachers, highlighting the approach's scalability and robustness for science-related reasoning.

Abstract

While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.

CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning

TL;DR

CoT-Evo introduces an evolutionary framework for distilling chain-of-thought reasoning tailored to scientific tasks. It builds a diverse candidate pool from multiple thinkers, augments knowledge, and refines CoTs through novelty-driven selection, reflective recombination, and mutation within an evaluation loop. A fitness function combining correctness, length appropriateness, and knowledge usage guides evolution, producing a compact, high-fidelity CoT dataset for training smaller models. Empirical results on BioProBench and ChemCoTBench show consistent improvements over traditional distillation baselines and competitive performance against larger LLM teachers, highlighting the approach's scalability and robustness for science-related reasoning.

Abstract

While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.
Paper Structure (55 sections, 16 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 55 sections, 16 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Long CoT Distillation
  4. Reasoning in Scientific Domains
  5. CoT-Evo: Evolutionary CoT Distillation
  6. Multi-thinker CoT Initialization
  7. CoT Generation
  8. Knowledge Augmentation
  9. Fitness Function
  10. Exact Match
  11. Length Appropriateness
  12. Knowledge Usage Correctness
  13. Novelty-Driven Candidate Selection
  14. Behavioral Embedding
  15. Novelty Score
  16. ...and 40 more sections

Figures (3)

  • Figure 1: Overview of CoT-Evo. (a) Multi-thinker CoT initialization constructs a diverse pool of candidates. (b) Novelty-driven candidate selection evaluates them via a composite fitness function and retains promising trajectories. (c) Reflective recombination and mutation generate new offspring CoTs through targeted operations. (d) These modules form the iterative CoT-Evo optimization loop, which evolves compact, accurate, and domain-reliable CoTs for downstream training.
  • Figure 2: Comparison of selection strategies. Left: Impact of selection strategies on BioProBench (average Pass@1 across all tasks). Right: Fitness dynamics across generation epochs, showing both average and maximum fitness.
  • Figure 3: Impact of budget (left) and population size (right) on CoT-Evo’s distillation performance.