SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded Interaction
Karthik Sunil, Aalok Thakkar
TL;DR
The paper addresses how to integrate LLMs into CS education without fostering dependency or shallow understanding. It proposes SocraticAI, a scaffolded tutoring system that enforces constraints (limited daily queries, guided dialogue, RAG grounding) and encourages reflection, backed by modular architecture and observability. Deployment in an undergraduate CS course shows improved question quality, substantial student reflection (≈75%), and reduced instructor workload, alongside identified vulnerabilities that guided iterative improvements. The work demonstrates that principled guardrails can cultivate AI literacy and scalable, conceptually deep learning while preserving curricular integrity.
Abstract
We present SocraticAI, a scaffolded AI tutoring system that integrates large language models (LLMs) into undergraduate Computer Science education through structured constraints rather than prohibition. The system enforces well-formulated questions, reflective engagement, and daily usage limits while providing Socratic dialogue scaffolds. Unlike traditional AI bans, our approach cultivates responsible and strategic AI interaction skills through technical guardrails, including authentication, query validation, structured feedback, and RAG-based course grounding. Initial deployment demonstrates that students progress from vague help-seeking to sophisticated problem decomposition within 2-3 weeks, with over 75% producing substantive reflections and displaying emergent patterns of deliberate, strategic AI use.