From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation

TL;DR

This work tackles unlabeled domain adaptation for LLMs by exploiting cognitive asymmetry: atomic sub-problems are reliably solved by LLMs even when holistic reasoning falters. It introduces Divergence-Guided Reasoning Curriculum (DGRC), which turns disagreements between teacher and student into two curricula—a high-quality atomic knowledge curriculum and a verified CoT curriculum—to guide atom-to-chain learning. The framework operates in three stages (divergence detection, curriculum generation, and student adaptation) and is validated across medical and legal domains, showing substantial gains for smaller models and strong generalization to unseen benchmarks. DGRC also demonstrates compatibility and potential synergy with reinforcement learning, offering a practical, efficient data engine for robust domain-specific reasoning without human annotations.

Abstract

Adapting Large Language Models (LLMs) to specialized domains without human-annotated data is a crucial yet formidable challenge. Widely adopted knowledge distillation methods often devolve into coarse-grained mimicry, where the student model inefficiently targets its own weaknesses and risks inheriting the teacher's reasoning flaws. This exposes a critical pedagogical dilemma: how to devise a reliable curriculum when the teacher itself is not an infallible expert. Our work resolves this by capitalizing on a key insight: while LLMs may exhibit fallibility in complex, holistic reasoning, they often exhibit high fidelity on focused, atomic sub-problems. Based on this, we propose Divergence-Guided Reasoning Curriculum (DGRC), which constructs a learning path from atomic knowledge to reasoning chains by dynamically deriving two complementary curricula from disagreements in reasoning pathways. When a student and teacher produce conflicting results, DGRC directs the teacher to perform a diagnostic analysis: it analyzes both reasoning paths to formulate atomic queries that target the specific points of divergence, and then self-answers these queries to create high-confidence atomic question-answer pairs. These pairs then serve a dual purpose: (1) providing an atomic curriculum to rectify the student's knowledge gaps, and (2) serving as factual criteria to filter the teacher's original reasoning chains, yielding a verified CoT curriculum that teaches the student how to integrate atomic knowledge into complete reasoning paths. Experiments across the medical and legal domains on student models of various sizes demonstrate the effectiveness of our DGRC framework. Notably, our method achieves a 7.76% relative improvement for the 1.5B student model in the medical domain over strong unlabeled baseline.

Figures (3)

• Figure 1: An illustration of the cognitive asymmetry in LLMs. The same LLM exhibits fallibility on a multi-step problem by incorrectly applying a flawed heuristic (left), yet demonstrates high fidelity when prompted with an atomic question that isolates the core conceptual error (right).
• Figure 2: The overview of the Divergence-Guided Reasoning Curriculum (DGRC) framework.
• Figure 3: Qualitative examples of student model errors. Token-level entropy is visualized by color intensity. The generated atomic question targets the specific reasoning error, whose source sentence is highlighted in red, which consistently aligns with a high-entropy region.