PedagoSense: A Pedology Grounded LLM System for Pedagogical Strategy Detection and Contextual Response Generation in Learning Dialogues

TL;DR

PedagoSense targets the gap in dialogue-based learning by jointly detecting pedagogical strategies and generating strategy-aligned tutor responses. The approach uses a two-stage pipeline: first a binary detector for the presence of pedagogy, then a fine-grained classifier for the specific strategy, complemented by an LLM that generates the tutor reply and a verification step to ensure alignment with the proposed strategy. Data augmentation with GPT-4o and DailyDialog improves training robustness, yielding state-of-the-art-like results on binary detection (F1 ≈ 98.5%) and meaningful gains in fine-grained classification with macro F1 improvements, while LLM evaluations reveal performance variability across models. The work bridges pedagogical theory and practical LLM-based response generation, offering a scalable path toward adaptive educational technologies and personalized tutoring experiences.

Abstract

This paper addresses the challenge of improving interaction quality in dialogue based learning by detecting and recommending effective pedagogical strategies in tutor student conversations. We introduce PedagoSense, a pedology grounded system that combines a two stage strategy classifier with large language model generation. The system first detects whether a pedagogical strategy is present using a binary classifier, then performs fine grained classification to identify the specific strategy. In parallel, it recommends an appropriate strategy from the dialogue context and uses an LLM to generate a response aligned with that strategy. We evaluate on human annotated tutor student dialogues, augmented with additional non pedagogical conversations for the binary task. Results show high performance for pedagogical strategy detection and consistent gains when using data augmentation, while analysis highlights where fine grained classes remain challenging. Overall, PedagoSense bridges pedagogical theory and practical LLM based response generation for more adaptive educational technologies.

Figures (5)

• Figure 1: Flow chart of the system
• Figure 2: Class Distribution: Before and After SMOTE
• Figure 3: Word Contributions to Classification
• Figure 4: Word Contributions to Classification
• Figure 5: Comparison of Pedagogical Strategies: Unbalanced vs Balanced.