IntelliChain: An Integrated Framework for Enhanced Socratic Method Dialogue with LLMs and Knowledge Graphs

TL;DR

The paper addresses the accuracy and reliability challenges of LLMs in domain-specific education by proposing IntelliChain, a framework that integrates LLMs with knowledge graphs and a multi-agent system to enable chain-of-thought dialogue for Socratic teaching. The approach structures dialogue strategy, knowledge-graph querying, and agent collaboration, with role-based interactions and reinforcement learning to adapt to learners. Key contributions include a math-focused knowledge graph, an integrated prompting mechanism, and empirical evaluation using the chicken-rabbit problem to show enhanced accuracy, depth, and personalization. This work demonstrates a practical path toward reliable, AI-driven personalized education and provides a blueprint for applying LLMs, knowledge graphs, and multi-agent systems in collaborative learning scenarios.

Abstract

With the continuous advancement of educational technology, the demand for Large Language Models (LLMs) as intelligent educational agents in providing personalized learning experiences is rapidly increasing. This study aims to explore how to optimize the design and collaboration of a multi-agent system tailored for Socratic teaching through the integration of LLMs and knowledge graphs in a chain-of-thought dialogue approach, thereby enhancing the accuracy and reliability of educational applications. By incorporating knowledge graphs, this research has bolstered the capability of LLMs to handle specific educational content, ensuring the accuracy and relevance of the information provided. Concurrently, we have focused on developing an effective multi-agent collaboration mechanism to facilitate efficient information exchange and chain dialogues among intelligent agents, significantly improving the quality of educational interaction and learning outcomes. In empirical research within the domain of mathematics education, this framework has demonstrated notable advantages in enhancing the accuracy and credibility of educational interactions. This study not only showcases the potential application of LLMs and knowledge graphs in mathematics teaching but also provides valuable insights and methodologies for the development of future AI-driven educational solutions.

Figures (2)

• Figure 1: The Framework of IntelliChain.
• Figure 2: Differential Teaching Outcomes Using the IntelliChain Framework Across Three System Configurations.