Never too Prim to Swim: An LLM-Enhanced RL-based Adaptive S-Surface Controller for AUVs under Extreme Sea Conditions

TL;DR

This work tackles robust, high-precision AUV control under extreme sea conditions by combining RL with an adaptive S-Surface controller and augmenting it with large-language-model guidance. The proposed three-module architecture—RL-based S-Surface control, LLM-enhanced joint optimization, and environment-aware simulation—learns task-level policies while ensuring accurate, disturbance-rejecting low-level signals via a sigmoid-based surface controller. By formulating the RL problem as a Markov decision process with control-affine dynamics and jointly optimizing reward structures and surface parameters, the approach achieves superior performance in 3D data collection and target-tracking tasks compared to PID and SMC baselines, under both standard and intensified disturbance scenarios. The results suggest significant practical potential for robust AUV operation in complex, unpredictable ocean environments, with future work focused on real-world implementation and sim2real transfer.

Abstract

The adaptivity and maneuvering capabilities of Autonomous Underwater Vehicles (AUVs) have drawn significant attention in oceanic research, due to the unpredictable disturbances and strong coupling among the AUV's degrees of freedom. In this paper, we developed large language model (LLM)-enhanced reinforcement learning (RL)-based adaptive S-surface controller for AUVs. Specifically, LLMs are introduced for the joint optimization of controller parameters and reward functions in RL training. Using multi-modal and structured explicit task feedback, LLMs enable joint adjustments, balance multiple objectives, and enhance task-oriented performance and adaptability. In the proposed controller, the RL policy focuses on upper-level tasks, outputting task-oriented high-level commands that the S-surface controller then converts into control signals, ensuring cancellation of nonlinear effects and unpredictable external disturbances in extreme sea conditions. Under extreme sea conditions involving complex terrain, waves, and currents, the proposed controller demonstrates superior performance and adaptability in high-level tasks such as underwater target tracking and data collection, outperforming traditional PID and SMC controllers.

Figures (6)

• Figure 1: Illustration of an AUV conducting underwater tasks using the proposed controller. The proposed controller utilizes an RL-based S-surface controller to enable effective control. LLMs assist the controller by optimizing the reward functions for RL training and tuning the parameters of the S-surface controller.
• Figure 2: The overall framework of our proposed controller, which comprises three modules: (A) RL-based S-Surface Controller Module. (B) LLM-Enhanced Iterative Joint Optimization Module. (C) Environment-Aware and Simulation Module.
• Figure 3: Parameters for yaw tracking controller and reward weights, along with 2D projections of AUV trajectories from the 3D data collection tasks during the LLM optimization phase.
• Figure 4: Comparative results of the S-surface controller in tracking reference signals taken from a target tracking task during the LLM optimization phase.
• Figure 5: Comparative results of three controllers tracking reference signals taken from a target tracking task under ES and VES conditions.