Never Too Rigid to Reach: Adaptive Virtual Model Control with LLM- and Lyapunov-Based Reinforcement Learning

TL;DR

This work tackles the brittleness of conventional robotic arm control by integrating Adaptive Virtual Model Control (VMC) with Lyapunov-stable, PPO-based reinforcement learning guided by Large Language Models (LLMs). It enables online adaptation of gains $K_{p,i}$, $K_{d,i}$ and coordination weights $(\alpha,\beta)$ while enforcing stability via a Lyapunov descent condition $\Delta L \le -c \|o - o^*\|^2$, and shaping rewards with LLM priors to penalize rigidity, unsafe behavior, or inefficiency. In simulations on a 7-DoF Panda arm, the method improves reaching accuracy to about $0.076$ m and reduces reach time, with multi-link configurations offering safety benefits at the cost of some accuracy. Overall, the combination of LLM guidance and Lyapunov-constrained adaptation yields a safe, adaptable framework that preserves the interpretability of VMC and enhances robustness in uncertain environments.

Abstract

Robotic arms are increasingly deployed in uncertain environments, yet conventional control pipelines often become rigid and brittle when exposed to perturbations or incomplete information. Virtual Model Control (VMC) enables compliant behaviors by embedding virtual forces and mapping them into joint torques, but its reliance on fixed parameters and limited coordination among virtual components constrains adaptability and may undermine stability as task objectives evolve. To address these limitations, we propose Adaptive VMC with Large Language Model (LLM)- and Lyapunov-Based Reinforcement Learning (RL), which preserves the physical interpretability of VMC while supporting stability-guaranteed online adaptation. The LLM provides structured priors and high-level reasoning that enhance coordination among virtual components, improve sample efficiency, and facilitate flexible adjustment to varying task requirements. Complementarily, Lyapunov-based RL enforces theoretical stability constraints, ensuring safe and reliable adaptation under uncertainty. Extensive simulations on a 7-DoF Panda arm demonstrate that our approach effectively balances competing objectives in dynamic tasks, achieving superior performance while highlighting the synergistic benefits of LLM guidance and Lyapunov-constrained adaptation.

Figures (7)

• Figure 1: Overall architecture of our proposed adaptive VMC framework, which integrates VMC with LLM- and Lyapunov-based RL.
• Figure 2: Simulation in Webots with a 7-DoF Franka Panda arm.
• Figure 3: Evaluation of our framework compared with the baseline.
• Figure 4: Quantitative comparison of our framework with the baseline.
• Figure 5: Learning curves of different VMC configurations under conventional RL and our proposed framework.