MetaCLASS: Metacognitive Coaching for Learning with Adaptive Self-regulation Support

TL;DR

MetaCLASS shifts AI tutoring from content delivery to metacognitive coaching by grounding interventions in self-regulated learning (SRL) theory via the Metacognitive Awareness Inventory (MAI) and an interpretable 11-move coach taxonomy, including No_intervention as a first-class action. It introduces a two-phase generation architecture: Phase 1 plans pedagogical trajectories conditioned on learner profiles, Phase 2 generates dialogue aligned to that plan, yielding 1,015 conversations (7,711 turns) across GSM8K, MATH, and AIME with turn-level MAI annotations. A Coach Move Prediction benchmark across nine LLMs reveals that even strong models struggle to choose appropriate metacognitive moves (best accuracy around 43.2%) and exhibit a pronounced compulsive intervention bias, under-predicting No_intervention and over-predicting high-intervention moves. The work demonstrates a critical gap between content-tutoring capability and metacognitive tutoring competence, offering MetaCLASS as a testbed and dataset for developing intelligent tutors that effectively foster self-regulated learning with contingent, restraint-aware guidance.

Abstract

Large language models can generate fluent explanations, but effective tutoring requires supporting the learner's thought process, not just delivering content. Metacognitive tutoring targets this gap by prompting planning, monitoring, debugging, and evaluation, and crucially, deciding when to be active versus minimally present, based on learner signals and trajectory. We introduce MetaCLASS, a learning-science grounded framework that formulates metacognitive tutoring as move selection over 11 interpretable actions aligned to self-regulated learning processes. MetaCLASS uses a two-phase framework that first plans a pedagogical trajectory conditioned on learner profiles (calibration, help-seeking) and then generates natural dialogue consistent with that plan. This yields a dataset of 1,015 conversations (7,711 turns) annotated with turn-level metacognitive labels, and validated for pedagogical contingency and trajectory adherence. We benchmark nine LLMs on predicting the next coach move given the problem and dialogue context. The best model achieves only 43.2\% accuracy, and models exhibit compulsive intervention bias: in turns where effective metacognitive tutoring requires silent (41.7\% of cases), models predict `no intervention' only 4.2\% of the time, while severely over-predicting high-intervention moves. These results show that traditional content-based tutoring ability does not translate to metacognitive tutoring competence, positioning MetaCLASS as a testbed for developing intelligent tutors that promote self-regulated learning.

Figures (4)

• Figure 1: An overview of the MetaCLASS Framework. (Left) We ground the framework in learning science by mapping MAI factors to LLM reasoning phases. (Center) We define a taxonomy of 11 interpretable coach moves, categorized by regulatory process. (Right) A two-phase generation architecture first plans a pedagogical trajectory together with learner profiles and problem analysis, which then guides the generation of metacognitive coaching dialogues.
• Figure 2: Category-level confusion matrix for Qwen3-80b-Think. DEBUGGING moves are predominantly misclassified as MONITORING (59%), suggesting the model struggles to distinguish exploratory probing from corrective intervention.
• Figure 3: Move-level confusion matrix for Qwen3-80b-Think. NI reaches only 29% accuracy, despite comprising 42% of ground truth, revealing compulsive intervention bias.
• Figure 4: Example of generated MetaCLASS conversation.