NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models

TL;DR

NIL addresses the challenge of acquiring physically plausible motor skills across diverse morphologies without curated expert data. It combines a frozen, pre-trained video diffusion model to generate on-demand reference videos with a discriminator-free imitation reward based on video-embedding similarity, segmentation IoU, and regularization, learned via an entropy-regularized off-policy actor-critic framework. The two-stage pipeline enables 3D policy learning from 2D generated data and demonstrates competitive performance across humanoid and quadruped robots, with improvements tied to diffusion model quality. This approach highlights the potential to replace data collection with data generation for imitation learning, enabling scalable skill acquisition across unconventional morphologies in robotics and animation.

Abstract

Acquiring physically plausible motor skills across diverse and unconventional morphologies-including humanoid robots, quadrupeds, and animals-is essential for advancing character simulation and robotics. Traditional methods, such as reinforcement learning (RL) are task- and body-specific, require extensive reward function engineering, and do not generalize well. Imitation learning offers an alternative but relies heavily on high-quality expert demonstrations, which are difficult to obtain for non-human morphologies. Video diffusion models, on the other hand, are capable of generating realistic videos of various morphologies, from humans to ants. Leveraging this capability, we propose a data-independent approach for skill acquisition that learns 3D motor skills from 2D-generated videos, with generalization capability to unconventional and non-human forms. Specifically, we guide the imitation learning process by leveraging vision transformers for video-based comparisons by calculating pair-wise distance between video embeddings. Along with video-encoding distance, we also use a computed similarity between segmented video frames as a guidance reward. We validate our method on locomotion tasks involving unique body configurations. In humanoid robot locomotion tasks, we demonstrate that 'No-data Imitation Learning' (NIL) outperforms baselines trained on 3D motion-capture data. Our results highlight the potential of leveraging generative video models for physically plausible skill learning with diverse morphologies, effectively replacing data collection with data generation for imitation learning.

Figures (10)

• Figure 1: NIL Overview: First, from a single frame and a textual prompt, a pre-trained video diffusion model generates a reference video. Reinforcement learning policies are then trained to mimic the generated video and control various robots without using any external data.
• Figure 2: NIL: No-data Imitation Learning consists two stages. Stage 1: Render the agent’s initial frame, remove the background, and generate a reference video using a pre-trained video diffusion model conditioned on the initial frame and a textual task description. Stage 2: Train a reinforcement learning agent in a physical simulation to imitate the generated video via a reward function comprising (1) video encoding similarity, (2) segmentation mask IoU, and (3) regularization for smooth behavior.
• Figure 3: All components
• Figure 4: Without regularization
• Figure 5: Without IoU