EasyUUV: An LLM-Enhanced Universal and Lightweight Sim-to-Real Reinforcement Learning Framework for UUV Attitude Control

TL;DR

The paper addresses the fragility of UUV attitude control under disturbances and model mismatch by proposing EasyUUV, a lightweight Sim2Real RL framework that couples a PPO-trained RL policy with a nonlinear adaptive A-S-Surface controller and multimodal LLM-based runtime tuning. Training occurs in a MuJoCo-based, domain-randomized simulation with GPU-accelerated parallelism, enabling zero-shot transfer of policies to a low-cost 6-DoF UUV hardware platform. Key contributions include a universal RL-based control architecture, LLM-driven online parameter adaptation, and extensive tank and sea-trial validation demonstrating robust, real-world performance without retraining. The work lowers hardware and development barriers while delivering reliable, high-precision attitude control across diverse underwater conditions.

Abstract

Despite recent advances in Unmanned Underwater Vehicle (UUV) attitude control, existing methods still struggle with generalizability, robustness to real-world disturbances, and efficient deployment. To address the above challenges, this paper presents EasyUUV, a Large Language Model (LLM)-enhanced, universal, and lightweight simulation-to-reality reinforcement learning (RL) framework for robust attitude control of UUVs. EasyUUV combines parallelized RL training with a hybrid control architecture, where a learned policy outputs high-level attitude corrections executed by an adaptive S-Surface controller. A multimodal LLM is further integrated to adaptively tune controller parameters at runtime using visual and textual feedback, enabling training-free adaptation to unmodeled dynamics. Also, we have developed a low-cost 6-DoF UUV platform and applied an RL policy trained through efficient parallelized simulation. Extensive simulation and real-world experiments validate the effectiveness and outstanding performance of EasyUUV in achieving robust and adaptive UUV attitude control across diverse underwater conditions. The source code is available from the following website: https://360zmem.github.io/easyuuv/

Figures (15)

• Figure 1: Illustration of our developed EasyUUV framework. EasyUUV is an LLM-enhanced universal and lightweight Sim2Real RL framework for UUV attitude control, which trains the expert policy via RL in parallelized simulation, while transferring it to a real UUV platform. A multimodal LLM agent further adapts controller parameters using dynamics and sensor feedback for robust performance.
• Figure 2: Architecture of the EasyUUV framework, which comprise three parts: (a) RL and A-S-Surface-based composite controller module; (b) Parallelized RL training simulation environment developed on Isaac Lab; and (c) Sim2Real deployment module for real-world adaption.
• Figure 3: Exploded view of our EasyUUV hardware platform.
• Figure 4: Experimental testbed for real-world validation of EasyUUV.
• Figure 5: Training curves of RL with different controllers in terms of reward and MSE. (Left): Reward curves. (Right): MSE curves.