MA-ROESL: Motion-aware Rapid Reward Optimization for Efficient Robot Skill Learning from Single Videos

TL;DR

This work addresses the inefficiencies and frame-sampling biases inherent in learning robot locomotion from single videos by introducing MA-ROESL, a framework that combines motion-aware frame selection with a three-phase reward-policy optimization pipeline. By prioritizing motion-salient frames and leveraging offline RL to rapidly evaluate VLM-generated rewards, MA-ROESL significantly accelerates training while maintaining faithful reproduction of diverse quadruped skills. The approach yields a ~68.7% average reduction in per-skill training time and demonstrates robust zero-shot sim-to-real transfer on a Unitree Go2, validating its practicality for scalable video-based skill learning. The results imply strong potential for leveraging large-scale video data to efficiently acquire robust locomotion skills in real-world robotics.

Abstract

Vision-language models (VLMs) have demonstrated excellent high-level planning capabilities, enabling locomotion skill learning from video demonstrations without the need for meticulous human-level reward design. However, the improper frame sampling method and low training efficiency of current methods remain a critical bottleneck, resulting in substantial computational overhead and time costs. To address this limitation, we propose Motion-aware Rapid Reward Optimization for Efficient Robot Skill Learning from Single Videos (MA-ROESL). MA-ROESL integrates a motion-aware frame selection method to implicitly enhance the quality of VLM-generated reward functions. It further employs a hybrid three-phase training pipeline that improves training efficiency via rapid reward optimization and derives the final policy through online fine-tuning. Experimental results demonstrate that MA-ROESL significantly enhances training efficiency while faithfully reproducing locomotion skills in both simulated and real-world settings, thereby underscoring its potential as a robust and scalable framework for efficient robot locomotion skill learning from video demonstrations.

Figures (7)

• Figure 1: Illustration of the skill learning problem from video demonstrations.
• Figure 2: The comparison of frame selection results: a) uniform sampling method. b) motion-aware frame selection method.
• Figure 3: Overview of MA-ROESL framework. 1) Phase 1: VLM-generated reward functions are used to train policies and collect an offline dataset for subsequent reward optimization. 2) Phase 2: Offline RL enables rapid reward optimization by relabeling rewards using the dataset and selecting the best-performing reward-policy pair. 3) Phase 3: The selected policy is fine-tuned online under the same reward to improve robustness in real-world deployment.
• Figure 4: Evaluation of motion pattern alignment using dynamic time warping (DTW) analysis.
• Figure 5: Contact patterns of the four limbs across different locomotion skills: a) Hop, b) Trot, c) Bound, and d) Pace, showing contact sequences of FL (Front Left), FR (Front Right), RL (Rear Left), and RR (Rear Right). Blue blocks indicate ground contact phases, while gaps denote swing phases.