MAS-KCL: Knowledge component graph structure learning with large language model-based agentic workflow

TL;DR

The paper tackles learning KC graphs to diagnose learner difficulties and tailor instruction. It introduces MAS-KCL, a large language model–driven multi-agent framework that uses a bidirectional edge-feedback mechanism and differential evolution with a multi-sub-population scheme to efficiently infer KC dependencies. Across 9 datasets (5 synthetic, 4 real-world) and multiple LLM variants, MAS-KCL achieves consistently lower loss than strong baselines and demonstrates strong generalization, with ablations confirming the value of each agent and the MAS component. The approach offers data-driven, causally grounded KC graphs that can guide targeted instructional interventions and support scalable, sustainable improvements in education. $AP$ controls elite retention and edge updates are guided by PFA/NFA in a dynamic loop, enabling adaptive structure learning with interpretability.

Abstract

Knowledge components (KCs) are the fundamental units of knowledge in the field of education. A KC graph illustrates the relationships and dependencies between KCs. An accurate KC graph can assist educators in identifying the root causes of learners' poor performance on specific KCs, thereby enabling targeted instructional interventions. To achieve this, we have developed a KC graph structure learning algorithm, named MAS-KCL, which employs a multi-agent system driven by large language models for adaptive modification and optimization of the KC graph. Additionally, a bidirectional feedback mechanism is integrated into the algorithm, where AI agents leverage this mechanism to assess the value of edges within the KC graph and adjust the distribution of generation probabilities for different edges, thereby accelerating the efficiency of structure learning. We applied the proposed algorithm to 5 synthetic datasets and 4 real-world educational datasets, and experimental results validate its effectiveness in learning path recognition. By accurately identifying learners' learning paths, teachers are able to design more comprehensive learning plans, enabling learners to achieve their educational goals more effectively, thus promoting the sustainable development of education.

Figures (5)

• Figure 1: The proposed agentic workflow for KC graph structure learning. This framework injects the emergent intelligence of large language models into the designed multi-agent system, enabling it to extract KCs from real-world learning data and construct KC graphs. The agentic workflow facilitates deep collaboration among learners, teachers, and AI agents, assisting teachers in identifying the root causes of learners' poor performance at the KC level, thereby providing support for the development of targeted instructional interventions.
• Figure 2: Game agent and feedback agents. The diagram displays the interaction between the Game Agent and the population, as well as the probability of adding new edges in the bidirectional feedback mechanism generated by the Feedback Agents.
• Figure 3: KC graph structure learning search process. Figure 1 shows the search mechanism for the graph KC Structure based on multi-sub-population collaboration.
• Figure 4: Results of ablation experiments on the multi-agent system component in MAS-KCL. These results illustrate the variation in $loss$ across different real-world educational datasets. Specifically, the comparison algorithm without the multi-agent system component is referred to as MAS-KCL (-MAS), represented by triangles, while the standard MAS-KCL is depicted by circles.
• Figure 5: Convergence analysis,which shows the convergence behavior and loss distribution of MAS-KCL. The centralized distribution of these results validates the strong convergence capability of the proposed method.