On the Role of Domain Experts in Creating Effective Tutoring Systems

TL;DR

This position paper argues that expert-specified knowledge is essential for effective AI in education, especially in high-stakes settings where reliability matters. It proposes two integrated approaches: augmenting explainable AI with domain-specific rules to automatically generate lessons, and using an expert-defined curriculum to structure a POMDP-based adaptive tutoring system. A pollinator identification case study demonstrates how elicited domain knowledge can be operationalized through curated rules and hierarchical concepts within learning workflows. The authors conclude with practical recommendations to empower experts, foster interdisciplinary collaboration, and broaden public education about the capabilities and limits of expert-informed educational AI.

Abstract

The role that highly curated knowledge, provided by domain experts, could play in creating effective tutoring systems is often overlooked within the AI for education community. In this paper, we highlight this topic by discussing two ways such highly curated expert knowledge could help in creating novel educational systems. First, we will look at how one could use explainable AI (XAI) techniques to automatically create lessons. Most existing XAI methods are primarily aimed at debugging AI systems. However, we will discuss how one could use expert specified rules about solving specific problems along with novel XAI techniques to automatically generate lessons that could be provided to learners. Secondly, we will see how an expert specified curriculum for learning a target concept can help develop adaptive tutoring systems, that can not only provide a better learning experience, but could also allow us to use more efficient algorithms to create these systems. Finally, we will highlight the importance of such methods using a case study of creating a tutoring system for pollinator identification, where such knowledge could easily be elicited from experts.

Figures (3)

• Figure 1: An example showing the application of our system.
• Figure 2: A hierarchical representation of concepts relevant to pollinator classification.
• Figure 3: An overview of our proposed tutoring system.