Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discovering the curriculum with AI: A proof-of-concept demonstration with an intelligent tutoring system for teaching project selection

Lovis Heindrich, Falk Lieder

TL;DR

This work addresses suboptimal real-world decision-making by leveraging AI to discover and teach cognitive strategies for project selection. It introduces MGPS, a metareasoning algorithm that derives resource-rational decision policies within a two-tier meta-level MDP and pairs it with MGPS Tutor, an intelligent tutoring system that trains humans to apply the discovered strategies. In simulations and a real-world-inspired training study, MGPS outperforms baselines and the MGPS Tutor significantly improves participants’ resource-rationality and adherence to the near-optimal strategy, demonstrating the feasibility of AI-powered boosting for educational contexts. The approach promises practical impact for executive decision-making and curriculum design, while acknowledging limitations around model fidelity, transfer, and real-world deployment, and outlining future work in Bayesian parameter estimation and broader educational deployment.

Abstract

The decisions of individuals and organizations are often suboptimal because fully rational decision-making is too demanding in the real world. Recent work suggests that some errors can be prevented by leveraging artificial intelligence to discover and teach clever heuristics. So far, this line of research has been limited to simplified, artificial decision-making tasks. This article is the first to extend this approach to a real-world decision problem, namely, executives deciding which project their organization should launch next. We develop a computational method (MGPS) that automatically discovers project selection strategies that are optimized for real people, and we develop an intelligent tutor that teaches the discovered project selection procedures. We evaluated MGPS on a computational benchmark and tested the intelligent tutor in a training experiment with two control conditions. MGPS outperformed a state-of-the-art method and was more computationally efficient. Moreover, people who practiced with our intelligent tutor learned significantly better project selection strategies than the control groups. These findings suggest that AI could be used to automate the process of discovering and formalizing the cognitive strategies taught by intelligent tutoring systems.

Discovering the curriculum with AI: A proof-of-concept demonstration with an intelligent tutoring system for teaching project selection

TL;DR

This work addresses suboptimal real-world decision-making by leveraging AI to discover and teach cognitive strategies for project selection. It introduces MGPS, a metareasoning algorithm that derives resource-rational decision policies within a two-tier meta-level MDP and pairs it with MGPS Tutor, an intelligent tutoring system that trains humans to apply the discovered strategies. In simulations and a real-world-inspired training study, MGPS outperforms baselines and the MGPS Tutor significantly improves participants’ resource-rationality and adherence to the near-optimal strategy, demonstrating the feasibility of AI-powered boosting for educational contexts. The approach promises practical impact for executive decision-making and curriculum design, while acknowledging limitations around model fidelity, transfer, and real-world deployment, and outlining future work in Bayesian parameter estimation and broader educational deployment.

Abstract

The decisions of individuals and organizations are often suboptimal because fully rational decision-making is too demanding in the real world. Recent work suggests that some errors can be prevented by leveraging artificial intelligence to discover and teach clever heuristics. So far, this line of research has been limited to simplified, artificial decision-making tasks. This article is the first to extend this approach to a real-world decision problem, namely, executives deciding which project their organization should launch next. We develop a computational method (MGPS) that automatically discovers project selection strategies that are optimized for real people, and we develop an intelligent tutor that teaches the discovered project selection procedures. We evaluated MGPS on a computational benchmark and tested the intelligent tutor in a training experiment with two control conditions. MGPS outperformed a state-of-the-art method and was more computationally efficient. Moreover, people who practiced with our intelligent tutor learned significantly better project selection strategies than the control groups. These findings suggest that AI could be used to automate the process of discovering and formalizing the cognitive strategies taught by intelligent tutoring systems.
Paper Structure (22 sections, 2 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 22 sections, 2 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Project selection
  4. Judge-advisor systems
  5. Resource rationality
  6. Strategy discovery methods
  7. Intelligent tutoring systems
  8. Cognitive tutors
  9. Formalizing optimal decision strategies for human project selection as the solution to a meta-level MDP
  10. A new metareasoning algorithm for discovering decision strategies for human project selection
  11. Improving human project selection
  12. MGPS Tutor: An intelligent tutor for teaching people how to select better projects
  13. Evaluating the effectiveness of MGPS Tutor in a training experiment
  14. A project-selection problem from the real world
  15. Methods of the experiment
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Example of the MGPS tutor offering a choice between requesting information from three different experts (highlighted in orange) in the simplified training task of deciding between two project alternatives.
  • Figure 2: Flowchart explaining the strategy discovered by MGPS.