ALIGNAgent: Adaptive Learner Intelligence for Gap Identification and Next-step guidance

TL;DR

Traditional education often relies on linear progression with limited diagnostic feedback, leading to unresolved misconceptions. ALIGNAgent presents a multi-agent framework that tightly couples knowledge estimation, concept-level skill-gap identification, and preference-aware resource recommendation in a closed adaptive loop, powered by LLM-based reasoning. The framework is validated on two undergraduate CS datasets, showing that a GPT-4o based setup achieves precision 0.87–0.90 and F1 0.84–0.87 in knowledge-proficiency estimation, while consistently identifying key gaps such as algorithm analysis and sequential circuits. This work demonstrates a scalable path toward end-to-end personalized learning in higher education, translating assessment data into targeted interventions that adapt as learners progress.”

Abstract

Personalized learning systems have emerged as a promising approach to enhance student outcomes by tailoring educational content, pacing, and feedback to individual needs. However, most existing systems remain fragmented, specializing in either knowledge tracing, diagnostic modeling, or resource recommendation, but rarely integrating these components into a cohesive adaptive cycle. In this paper, we propose ALIGNAgent (Adaptive Learner Intelligence for Gap Identification and Next-step guidance), a multi-agent educational framework designed to deliver personalized learning through integrated knowledge estimation, skill-gap identification, and targeted resource recommendation.ALIGNAgent begins by processing student quiz performance, gradebook data, and learner preferences to generate topic-level proficiency estimates using a Skill Gap Agent that employs concept-level diagnostic reasoning to identify specific misconceptions and knowledge deficiencies. After identifying skill gaps, the Recommender Agent retrieves preference-aware learning materials aligned with diagnosed deficiencies, implementing a continuous feedback loop where interventions occur before advancing to subsequent topics. Extensive empirical evaluation on authentic datasets from two undergraduate computer science courses demonstrates ALIGNAgent's effectiveness, with GPT-4o-based agents achieving precision of 0.87-0.90 and F1 scores of 0.84-0.87 in knowledge proficiency estimation validated against actual exam performance.

Figures (6)

• Figure 1: Traditional linear instruction versus AI-assisted personalized learning with skill-gap identification and targeted resource recommendation.
• Figure 2: Overview of $\mathsf{ALIGNAgent}$. Student data (preferences, quiz questions, activity logs, gradebook) is processed by the Skill Gap Agent to generate knowledge proficiency estimates and identify skill gaps. The Recommender Agent then retrieves preference-aligned learning materials based on these gaps, while the Summary Agent synthesizes diagnostic insights into actionable learner feedback.
• Figure 3: Questions distribution by topic and difficulty level for (a) COP3415 (b) CDA2108
• Figure 4: Comparison of question labeling techniques across different LLMs for (a) COP3415 Data Structures and (b) CDA2108 Fundamentals of Computer Systems datasets. GPT-4o shows superior performance across all evaluation metrics.
• Figure 5: Agent performance comparison across different LLMs for (a) COP3415 Data Structures and (b) CDA2108 Fundamentals of Computer Systems. GPT-4o consistently outperforms Claude 3.5 Sonnet and LLaMA 3.3 across all metrics.