WisdomBot: Tuning Large Language Models with Artificial Intelligence Knowledge

TL;DR

WisdomBot addresses core limitations of general LLMs in education by grounding training data in textbook-level knowledge concepts and Bloom’s Taxonomy, and by employing self-instruction to create extensive, domain-focused instruction data. During inference, it uses local knowledge base retrieval and external search augmentation to enhance factual accuracy and professional quality. Evaluations on Chinese LLMs (Chinese-LLaMA-Alpaca and Qwen-7B-Chat) show WisdomBot outperforms baselines across education-specific tasks and C-Eval benchmarks, with notable gains in creativity, personalization, and logical reasoning. This work demonstrates a scalable approach to tailoring LLMs for educational use with practical retrieval tooling to support reliable, context-aware responses.

Abstract

Large language models (LLMs) have emerged as powerful tools in natural language processing (NLP), showing a promising future of artificial generated intelligence (AGI). Despite their notable performance in the general domain, LLMs have remained suboptimal in the field of education, owing to the unique challenges presented by this domain, such as the need for more specialized knowledge, the requirement for personalized learning experiences, and the necessity for concise explanations of complex concepts. To address these issues, this paper presents a novel LLM for education named WisdomBot, which combines the power of LLMs with educational theories, enabling their seamless integration into educational contexts. To be specific, we harness self-instructed knowledge concepts and instructions under the guidance of Bloom's Taxonomy as training data. To further enhance the accuracy and professionalism of model's response on factual questions, we introduce two key enhancements during inference, i.e., local knowledge base retrieval augmentation and search engine retrieval augmentation during inference. We substantiate the effectiveness of our approach by applying it to several Chinese LLMs, thereby showcasing that the fine-tuned models can generate more reliable and professional responses.

Figures (9)

• Figure 1: Limitations of general LLMs in education: (a) comprehension ability, (b) out-of-date knowledge, (c) personalized ability, (d) Chinese proficiency, (e) logical reasoning ability.
• Figure 2: Training pipeline. We collect knowledge concepts and instructions under the guidance of textbooks, Bloom’s Taxonomy and strong LLMs, serving as instruction-tuning data to transform general LLMs to educational LLMs. During inference, we construct a local knowledge base based on the textbook, incorporating search engine capabilities for retrieval enhancement.
• Figure 3: Demonstrations of the dataset.
• Figure 4: Distribution of test data.
• Figure 5: Human evaluation of whether WisdomBot outperforms Chinese-Alpaca-7B.