IB-GRPO: Aligning LLM-based Learning Path Recommendation with Educational Objectives via Indicator-Based Group Relative Policy Optimization

TL;DR

IB-GRPO presents an indicator-guided, Pareto-aware framework for aligning LLM-based Learning Path Recommendation with educational objectives such as maximizing learning effect while following Zone of Proximal Development guidance. It combines a hybrid expert warm-start (GA plus teacher RL) with an on-policy IB-GRPO stage that uses an $I_{epsilon+}$-based group-relative advantage to optimize a vector of objectives Ep, S_ZPD, R_Len, and D_Div without manual scalarization. The method explicitly models ZPD-based difficulty, length constraints, and trajectory diversity, and demonstrates Pareto-balanced gains on ASSIST09 and Junyi via the KES simulator with a Qwen2.5-7B backbone, outperforming RL and prompting-based baselines. This approach provides a scalable, pedagogy-aligned pathway for long-horizon LPR, with practical impact for adaptive learning systems seeking nuanced trade-offs among learning outcomes, difficulty, pacing, and diversity.

Abstract

Learning Path Recommendation (LPR) aims to generate personalized sequences of learning items that maximize long-term learning effect while respecting pedagogical principles and operational constraints. Although large language models (LLMs) offer rich semantic understanding for free-form recommendation, applying them to long-horizon LPR is challenging due to (i) misalignment with pedagogical objectives such as the Zone of Proximal Development (ZPD) under sparse, delayed feedback, (ii) scarce and costly expert demonstrations, and (iii) multi-objective interactions among learning effect, difficulty scheduling, length controllability, and trajectory diversity. To address these issues, we propose IB-GRPO (Indicator-Based Group Relative Policy Optimization), an indicator-guided alignment approach for LLM-based LPR. To mitigate data scarcity, we construct hybrid expert demonstrations via Genetic Algorithm search and teacher RL agents and warm-start the LLM with supervised fine-tuning. Building on this warm-start, we design a within-session ZPD alignment score for difficulty scheduling. IB-GRPO then uses the $I_{ε+}$ dominance indicator to compute group-relative advantages over multiple objectives, avoiding manual scalarization and improving Pareto trade-offs. Experiments on ASSIST09 and Junyi using the KES simulator with a Qwen2.5-7B backbone show consistent improvements over representative RL and LLM baselines.

Figures (5)

• Figure 1: Overall framework of IB-GRPO. Stage I (top): Hybrid expert data synthesis (GA search + offline RL teachers) produces demonstration paths for SFT, yielding a model $\mu_{\mathrm{sft}}$. Stage II (bottom): Starting from the SFT model, we sample $K$ candidate paths per prompt, compute multi-objective rewards, and derive group-relative advantages via the $I_{\epsilon+}$-based fitness to update the policy toward the Pareto frontier.
• Figure 2: Pareto analysis of data synthesis strategies on ASSIST09 and Junyi datasets.
• Figure 3: Ablation of the ZPD reward across path lengths ($L\in{5,10,20}$), evaluated by mean $E_p$ on ASSIST09 and Junyi.
• Figure 4: The axes correspond to Learning Effect ($E_p$), ZPD alignment score ($S_{ZPD}$), trajectory-level diversity ($\mathrm{Div}_{\text{path}}$), and length satisfaction score ($\mathrm{LenScore}$).
• Figure 5: Trajectory comparison of recommended concept difficulty relative to the ZPD band. The blue area visualizes the ZPD region (centered on $z(a)$ with $\pm$$\sigma$ band).