Pedagogical Alignment for Vision-Language-Action Models: A Comprehensive Framework for Data, Architecture, and Evaluation in Education

TL;DR

The paper tackles the challenge of deploying vision-language-action (VLA) robots for education in resource-constrained classrooms by introducing Pedagogical VLA Framework, a lightweight, pedagogy-aligned approach. It combines text healing to restore language output, distillation of pedagogical knowledge from large language models, safety training for human-in-the-loop environments, and a multi-dimensional evaluation that includes teacher feedback and LLM-based text quality judgments. Across five science demonstrations, the framework achieves comparable task performance to baselines while yielding significantly more pedagogically rich and safety-aware explanations, demonstrating the feasibility of interpretable, education-focused robotics in real classrooms. The work contributes a concrete data-and-architecture pipeline, an evaluation framework with qualitative and quantitative metrics, and empirical evidence that pedagogy-focused text generation can be effectively integrated without sacrificing safety or practicality, making classroom deployment more viable. It also provides a publicly released model and dataset to spur further advancement at the intersection of robotics and education.

Abstract

Science demonstrations are important for effective STEM education, yet teachers face challenges in conducting them safely and consistently across multiple occasions, where robotics can be helpful. However, current Vision-Language-Action (VLA) models require substantial computational resources and sacrifice language generation capabilities to maximize efficiency, making them unsuitable for resource-constrained educational settings that require interpretable, explanation-generating systems. We present \textit{Pedagogical VLA Framework}, a framework that applies pedagogical alignment to lightweight VLA models through four components: text healing to restore language generation capabilities, large language model (LLM) distillation to transfer pedagogical knowledge, safety training for educational environments, and pedagogical evaluation adjusted to science education contexts. We evaluate Pedagogical VLA Framework across five science demonstrations spanning physics, chemistry, biology, and earth science, using an evaluation framework developed in collaboration with science education experts. Our evaluation assesses both task performance (success rate, protocol compliance, efficiency, safety) and pedagogical quality through teacher surveys and LLM-as-Judge assessment. We additionally provide qualitative analysis of generated texts. Experimental results demonstrate that Pedagogical VLA Framework achieves comparable task performance to baseline models while producing contextually appropriate educational explanations.

Figures (4)

• Figure 1: Robot science demonstrations with pedagogically-aligned explanations connecting actions to learning objectives (left, center), and safety-aware behavior enabling immediate halt upon detecting human presence in the workspace (right).
• Figure 2: Overview of the Pedagogical VLA Framework. The framework integrates data collection via teleoperation with LLM-generated pedagogical text, extends SmolVLA with a text decoder trained for safety awareness and pedagogical alignment, and evaluates both task execution and text quality through human experts and LLM-as-Judge.
• Figure 3: Descriptive statistics of generated text. The left panel shows average text length by model, where Pedagogical VLA generates 10.2$\times$ longer utterances (697 vs 68 characters). The center panel compares text volume by experiment between Pedagogical VLA (blue) and Text-SmolVLA (orange). The right panel shows Pedagogical VLA category distribution with Procedural Support (68.9%) and Conceptual Support (28.9%) as dominant categories.
• Figure 4: Main analysis results. The left panel shows experiment-specific subcategory patterns where EM Induction emphasizes Experiment Goal (52%) and Observation-Concept (40%), while Rock Classification emphasizes Procedure Guidance (90%). The center panel presents Safety Management by experiment and hand condition, revealing inherent safety awareness in Flame Test (3.2% without hand) versus reactive safety in other experiments (0% without hand). The right panel displays keyword frequency patterns by experiment showing distinct pedagogical emphases across tasks.