Cross-platform Learning-based Fault Tolerant Surfacing Controller for Underwater Robots

TL;DR

The paper tackles fault-tolerant surfacing for underwater robots under actuator failures across diverse platform geometries. It employs a PPO-based reinforcement learning framework enhanced with LSTM to learn a robust surfacing policy that does not require identifying failed actuators, and introduces cross-platform transfer by sharing early LSTM layers across designs. Across hovering AUVs, torpedo AUVs, and fin-actuated U-CAT, the method demonstrates improved stability and success rates, including a real-world 24/28 success in pool tests (85.7%) versus 8/14 (57.1%) for a baseline. The work offers a scalable approach for multi-platform underwater robotics, with Sim2Real validation and open-source tooling to support broader adoption.

Abstract

In this paper, we propose a novel cross-platform fault-tolerant surfacing controller for underwater robots, based on reinforcement learning (RL). Unlike conventional approaches, which require explicit identification of malfunctioning actuators, our method allows the robot to surface using only the remaining operational actuators without needing to pinpoint the failures. The proposed controller learns a robust policy capable of handling diverse failure scenarios across different actuator configurations. Moreover, we introduce a transfer learning mechanism that shares a part of the control policy across various underwater robots with different actuators, thus improving learning efficiency and generalization across platforms. To validate our approach, we conduct simulations on three different types of underwater robots: a hovering-type AUV, a torpedo shaped AUV, and a turtle-shaped robot (U-CAT). Additionally, real-world experiments are performed, successfully transferring the learned policy from simulation to a physical U-CAT in a controlled environment. Our RL-based controller demonstrates superior performance in terms of stability and success rate compared to a baseline controller, achieving an 85.7 percent success rate in real-world tests compared to 57.1 percent with a baseline controller. This research provides a scalable and efficient solution for fault-tolerant control for diverse underwater platforms, with potential applications in real-world aquatic missions.

Figures (6)

• Figure 1: Overview of the proposed method. The task to be accomplished is that when at least one or more actuators are in a failed state, the robot will use the sensor information to surface using the actuators that are operational. In this case, the identification of the failed actuator is not required. In addition, by transferring the first layer of the policy network, part of the RL model is shared between multiple different platforms.
• Figure 2: Three different platform models and actuator configurations. (A) Hovering type AUV with 8 thrusters. 8 thrusters are arranged symmetrically and the arrow on the thruster indicates the positive direction of output. (B) Torpedo-shaped AUV. Green area indicates the rudders, which can change angle as direction of red arrows. Red indicates thrusters for thrust in the surge direction, and arrows indicate positive direction of output. (C) Turtle-shaped robot (U-CAT). The green part shows the fins, which are paddled to generate thrust. Dynamics of the fin is shown in the lower right.
• Figure 3: Schematic diagram of an Actor-Critic based RL controller for turtle-shaped robot (U-CAT) using simulated dynamics. The left side shows the procedure for training the controller in the simulator, and the right side shows how to apply the controller to the real robot (Sim2Real).
• Figure 4: Success rate transitions in the last 100 trials during training of the model (Vanilla and first layer transfer).
• Figure 5: (a) The test setup in a swimming pool (Keila, Estonia). (b) The ArUco markers and dimensions of the workspace.