CompliantVLA-adaptor: VLM-Guided Variable Impedance Action for Safe Contact-Rich Manipulation

TL;DR

This work addresses the safety gap in vision-language-action (VLA) robotic manipulation by introducing a CompliantVLA-adaptor that grounds high-level semantic planning in force-aware, compliant execution. The adaptor uses a frozen vision-language model (VLM) to infer context and generate context-sensitive impedance parameters for a variable impedance controller (VIC), while continually regulating interaction forces with real-time feedback. It integrates contact-phase recognition and anisotropic impedance tuning across three time scales (≈1 Hz for impedance, ≈3 Hz for actions, and ≈1000 Hz for low-level control), enabling safe interaction during complex contact-rich tasks. Results in simulation and on real hardware show improved task success and reduced force violations compared to state-of-the-art VLA baselines, demonstrating a practical path toward safer and more generalizable contact-rich manipulation. The approach provides a lightweight, plug-and-play safety layer that links semantic understanding with compliant physical interaction, with potential for broader deployment as VLMs and onboard latency improve.

Abstract

We propose a CompliantVLA-adaptor that augments the state-of-the-art Vision-Language-Action (VLA) models with vision-language model (VLM)-informed context-aware variable impedance control (VIC) to improve the safety and effectiveness of contact-rich robotic manipulation tasks. Existing VLA systems (e.g., RDT, Pi0, OpenVLA-oft) typically output position, but lack force-aware adaptation, leading to unsafe or failed interactions in physical tasks involving contact, compliance, or uncertainty. In the proposed CompliantVLA-adaptor, a VLM interprets task context from images and natural language to adapt the stiffness and damping parameters of a VIC controller. These parameters are further regulated using real-time force/torque feedback to ensure interaction forces remain within safe thresholds. We demonstrate that our method outperforms the VLA baselines on a suite of complex contact-rich tasks, both in simulation and on real hardware, with improved success rates and reduced force violations. The overall success rate across all tasks increases from 9.86\% to 17.29\%, presenting a promising path towards safe contact-rich manipulation using VLAs. We release our code, prompts, and force-torque-impedance-scenario context datasets at https://sites.google.com/view/compliantvla.

Figures (5)

• Figure 1: Existing VLA systems lack force-awareness, leading to unsafe interactions in physical tasks involving contact or uncertainty. We see this challenge as a promising direction for safer deployment of VLA systems.
• Figure 2: Overview of the CompliantVLA-adaptor. A VLM processes visual observations, language instructions, and real-time force feedback $\mathcal{F}$ to generate context-aware impedance parameters $\mathcal{K, D}$. These parameters modulate a variable impedance controller(VIC) that executes actions generated by a VLA model, ensuring safe and adaptive contact-rich manipulation.
• Figure 3: Contact-rich tasks in simulation.
• Figure 4: Evaluation results of task success rates under contact force constraint across 8 related tasks in simulation. Title of each bar diagram indicates a different setup, where "T" with a value denotes task number, see its description in Tab. \ref{['tab:task_list_simulation']}, and the suffix "-R", "-P" and "-O" means using RDT, Pi0 and OpenVLA-oft model, respectively. different colors indicate the run with and without our CompliantVLA-adaptor. The y-axis indicates the task success rates under the contact force threshold of 30N. The results show that the baseline VLA models exhibit highly unstable performance across the task suite, even worse 0% in some tasks. Our CompliantVLA-adaptor improves performance across most tasks, demonstrating its effectiveness.
• Figure 5: Real-world experiment evaluation: Stiffness regulation and measured force during the pushing task.