ActDistill: General Action-Guided Self-Derived Distillation for Efficient Vision-Language-Action Models

TL;DR

ActDistill tackles the computational burden of Vision-Language-Action models in embodied manipulation by transferring action-centric semantics from a large teacher to a light-weight student via graph-structured encapsulation and an action-guided dynamic router. The method builds sparse, action-aligned graphs from teacher representations to produce action priors, while the student learns to reconstruct these semantics with selective computation. A joint objective combines semantic alignment and strict action consistency, and a router gates layers during inference to meet real-time constraints. Across autoregressive and diffusion-based backbones on LIBERO and SIMPLER benchmarks, ActDistill delivers comparable or superior task performance with over 50% reductions in FLOPs and up to 1.67× speedups, offering a general pathway to efficient embodied intelligence.

Abstract

Recent Vision-Language-Action (VLA) models have shown impressive flexibility and generalization, yet their deployment in robotic manipulation remains limited by heavy computational overhead and inference latency. In this work, we present ActDistill, a general action-guided self-derived distillation framework that transfers the action prediction capability of any existing VLA model to a lightweight counterpart. Unlike previous efficiency strategies that primarily emphasize vision-language correlations, ActDistill leverages action priors to guide knowledge transfer and model compression, achieving action-oriented efficiency for VLA models. Specifically, we employ a well-trained VLA model as the teacher and introduce a graph-structured encapsulation strategy to explicitly model the hierarchical evolution of action prediction. The student model, derived from the graph-encapsulated teacher, is further equipped with a dynamic router that adaptively selects computation paths based on action prediction demands, guided by hierarchical graph-informed supervision to ensure smooth and efficient evolution. During inference, graph-related auxiliary components are removed, allowing the student to execute only dynamically routed layers and predict high-precision actions with minimal computation and latency. Experiments on embodied benchmarks demonstrate that ActDistill achieves comparable or superior performance to full-scale VLA models while reducing computation by over 50% with up to 1.67 times speedup, thereby establishing a general paradigm toward efficient embodied intelligence.

Figures (7)

• Figure 1: Comparison between previous efficiency VLA strategies and our proposed ActDistill.
• Figure 2: Overview of our ActDistill framework.
• Figure 3: Performance-efficiency trade-off across different layer skipping configurations.
• Figure 4: Visualization of layer-wise activation frequency across the VLA backbone.
• Figure 5: Comparison of manipulation trajectories between the original and ActDistill-optimized VLA models.