EduPlanner: LLM-Based Multi-Agent Systems for Customized and Intelligent Instructional Design

TL;DR

EduPlanner tackles the challenge of automated, customized instructional design by introducing an LLM-based multi-agent framework with three collaborating agents: Evaluator, Optimizer, and Analyst. It models student backgrounds via a Skill-Tree and uses a CIDDP evaluation module to drive iterative optimization, yielding personalized and higher-quality instructional designs for curriculum and learning activities. Empirical results on GSM8K and Algebra datasets, along with ablation studies, demonstrate the framework’s effectiveness and the importance of the Skill-Tree and Analyst components. The work advances scalable, AI-assisted instructional design with potential for broad impact in personalized education and educational technology adoption.

Abstract

Large Language Models (LLMs) have significantly advanced smart education in the Artificial General Intelligence (AGI) era. A promising application lies in the automatic generalization of instructional design for curriculum and learning activities, focusing on two key aspects: (1) Customized Generation: generating niche-targeted teaching content based on students' varying learning abilities and states, and (2) Intelligent Optimization: iteratively optimizing content based on feedback from learning effectiveness or test scores. Currently, a single large LLM cannot effectively manage the entire process, posing a challenge for designing intelligent teaching plans. To address these issues, we developed EduPlanner, an LLM-based multi-agent system comprising an evaluator agent, an optimizer agent, and a question analyst, working in adversarial collaboration to generate customized and intelligent instructional design for curriculum and learning activities. Taking mathematics lessons as our example, EduPlanner employs a novel Skill-Tree structure to accurately model the background mathematics knowledge of student groups, personalizing instructional design for curriculum and learning activities according to students' knowledge levels and learning abilities. Additionally, we introduce the CIDDP, an LLM-based five-dimensional evaluation module encompassing clarity, Integrity, Depth, Practicality, and Pertinence, to comprehensively assess mathematics lesson plan quality and bootstrap intelligent optimization. Experiments conducted on the GSM8K and Algebra datasets demonstrate that EduPlanner excels in evaluating and optimizing instructional design for curriculum and learning activities. Ablation studies further validate the significance and effectiveness of each component within the framework. Our code is publicly available at https://github.com/Zc0812/Edu_Planner

Figures (7)

• Figure 1: The proportion of people who consider various factors critical in the task of generating lesson plans. Taking mathematics lessons as our example, in EduPlanner, we model the student situation using a newly proposed Skill-Tree ability module. We iteratively evaluate and optimize the teaching method and teaching content through the adversarial collaboration between the Evaluator Agent and the Optimizer Agent. Additionally, an Analyst Agent is employed to analyze error-prone examples and incorporate them into the subsequent lesson plan. EduPlanner effectively addresses the most pressing challenges in lesson planning.
• Figure 2: Examples of datasets. On the left is GSM8K cobbe2021training and on the right is Algebra he2023solving. GSM8K is a dataset consisting of 8.5K high-quality, language-diversified elementary school mathematics word problems released by OpenAI. Algebra is a dataset comprised of 222 examples related to algebraic equations.
• Figure 3: EduPlanner: LLM-Multi-Agent-Based Instructional Design Generation Framework. When the optimization is completed, the instructional design for curriculum and learning activities with the highest score in the queue is output, and its content mainly includes two parts: knowledge point explanation and question explanation.
• Figure 4: Skill-Tree structure models the students' diverse knowledge background through five principal abilities, outputs the students' Skill-Tree level, and provides it to the agents so that the agents can capture the knowledge background of different students and ensure that targeted instructional design for curriculum and learning activities can be generated.
• Figure 5: In this figure, we obtained the optimization process curves of our framework and He's method he2024evaluating under the premise that the hyperparameters are consistent with the initial instructional design for curriculum and learning activities. Our framework has a higher score in the instructional design than He's he2024evaluating method, and the optimization process is smoother.