Gentle Manipulation Policy Learning via Demonstrations from VLM Planned Atomic Skills

TL;DR

Long-horizon manipulation tasks require delicate contact interactions and extensive data; this paper presents a force-aware framework that uses Visual Language Model (VLM) planning to decompose tasks into tactile atomic skills trained in simulation, followed by Visual-Tactile Diffusion Policy (VT-DP) distillation into end-to-end policies. The approach integrates a tactile skill library, SAC-based force-aware reinforcement learning, VLM-driven planning (VASK), and diffusion-based imitation learning, enabling scalable data generation and robust generalization. Key contributions include the VASK data-generation pipeline, the VT-DP distillation framework, comprehensive ablations on planners and force components, and real-world deployment via a digital twin to bridge simulation and reality. The results demonstrate improved stability, gentleness, and generalization across varied long-horizon tasks while reducing manual demonstration costs.

Abstract

Autonomous execution of long-horizon, contact-rich manipulation tasks traditionally requires extensive real-world data and expert engineering, posing significant cost and scalability challenges. This paper proposes a novel framework integrating hierarchical semantic decomposition, reinforcement learning (RL), visual language models (VLMs), and knowledge distillation to overcome these limitations. Complex tasks are decomposed into atomic skills, with RL-trained policies for each primitive exclusively in simulation. Crucially, our RL formulation incorporates explicit force constraints to prevent object damage during delicate interactions. VLMs perform high-level task decomposition and skill planning, generating diverse expert demonstrations. These are distilled into a unified policy via Visual-Tactile Diffusion Policy for end-to-end execution. We conduct comprehensive ablation studies exploring different VLM-based task planners to identify optimal demonstration generation pipelines, and systematically compare imitation learning algorithms for skill distillation. Extensive simulation experiments and physical deployment validate that our approach achieves policy learning for long-horizon manipulation without costly human demonstrations, while the VLM-guided atomic skill framework enables scalable generalization to diverse tasks.

Figures (6)

• Figure 1: Our pipeline trains robotic arm tactile skills through force-constrained reinforcement learning in simulation. Visual Language Models then plan task sequences by interpreting visual scenes and language instructions to generate expert demonstrations. These demonstrations are distilled into contact-aware manipulation policies via visual-tactile Diffusion Policy, enabling end-to-end execution of long-horizon tasks from multi-modal point cloud inputs.
• Figure 2: Schematic diagram of contact force of tactile atomic skills.
• Figure 3: Framework diagram of VASK. The VLM receives system prompts, natural language task descriptions, and RGB images, and integrates this information with atomic skills to guide the agent in completing manipulation tasks and collecting raw trajectory data.
• Figure 4: Point cloud sequence during a long-horizon manipulation task. The boxed region in the upper-left corner of the visual point cloud shows the tactile point cloud. The legend indicates the contact depth between the sensor and the object. Note that the point cloud data itself is uncolored; colors are applied solely for visualization purposes.
• Figure 5: Contact force distribution over ten trials for each of the four manipulation tasks. Blue boxplots show policies trained without force-related penalties, while red boxplots show policies trained with force-aware rewards.