UCO: A Multi-Turn Interactive Reinforcement Learning Method for Adaptive Teaching with Large Language Models
Shouang Wei, Min Zhang, Xin Lin, Bo Jiang, Kun Kuang, Zhongxiang Dai
TL;DR
This work presents Unidirectional Cognitive Optimization (UCO), a cognition-aware, multi-turn reinforcement learning framework for adaptive teaching with large language models. By coupling a Progress Reward that tracks genuine cognitive advancement with a Scaffold Reward that dynamically locates and operates within the Zone of Proximal Development, UCO guides the teacher policy through Grouped Rollout Policy Optimization (GRPO). Empirical results on BigMath and MathTutorBench show that UCO achieves a superior balance across teaching effectiveness, solution leakage, and pedagogical quality, outperforming comparable-scale baselines and approaching advanced closed-source models. The approach emphasizes representing and leveraging students' evolving cognitive states to tailor scaffolding, with potential applicability to broader educational domains beyond mathematics.
Abstract
Large language models (LLMs) are shifting from answer providers to intelligent tutors in educational settings, yet current supervised fine-tuning methods only learn surface teaching patterns without dynamic adaptation capabilities. Recent reinforcement learning approaches address this limitation but face two critical challenges. First, they evaluate teaching effectiveness solely based on whether students produce correct outputs, unable to distinguish whether students genuinely understand or echo teacher-provided answers during interaction. Second, they cannot perceive students' evolving cognitive states in real time through interactive dialogue, thus failing to adapt teaching strategies to match students' cognitive levels dynamically. We propose the Unidirectional Cognitive Optimization (UCO) method to address these challenges. UCO uses a multi-turn interactive reinforcement learning paradigm where the innovation lies in two synergistic reward functions: the Progress Reward captures students' cognitive advancement, evaluating whether students truly transition from confusion to comprehension, while the Scaffold Reward dynamically identifies each student's Zone of Proximal Development (ZPD), encouraging teachers to maintain productive teaching within this zone. We evaluate UCO by comparing it against 11 baseline models on BigMath and MathTutorBench benchmarks. Experimental results demonstrate that our UCO model outperforms all models of equivalent scale and achieves performance comparable to advanced closed-source models. The code and data are available at https://github.com/Mind-Lab-ECNU/UCO.