Underwater Soft Fin Flapping Motion with Deep Neural Network Based Surrogate Model

TL;DR

This work tackles the challenge of controlling soft-fin underwater robots, whose nonlinear, flexible dynamics complicate precise force control. It introduces a DNN-based surrogate (PosNet for motor motion and ForceNet for forces) trained on real data, and trains an RL controller inside this surrogate, using a Sobolev-norm-based reward and a grid-switching strategy to specialize models for different force references. The approach enables Real2Sim2Real transfer to a real four-flipper actuator, achieving improved force tracking over a single RL model and reducing the need for extensive live testing. The contributions offer a scalable framework for robust, data-driven control of soft actuators in challenging underwater environments, with potential extensions to multi-fin and body-frame force control.

Abstract

This study presents a novel framework for precise force control of fin-actuated underwater robots by integrating a deep neural network (DNN)-based surrogate model with reinforcement learning (RL). To address the complex interactions with the underwater environment and the high experimental costs, a DNN surrogate model acts as a simulator for enabling efficient training for the RL agent. Additionally, grid-switching control is applied to select optimized models for specific force reference ranges, improving control accuracy and stability. Experimental results show that the RL agent, trained in the surrogate simulation, generates complex thrust motions and achieves precise control of a real soft fin actuator. This approach provides an efficient control solution for fin-actuated robots in challenging underwater environments.

Figures (7)

• Figure 1: An example of the fin actuated robot with four individually actuated soft flippers. $F_A$ is the overall vector of the generated force in the direction of the fin's $x$-axis.
• Figure 2: Method overview. The surrogate simulation model is trained with a dataset that includes fin motion and force measured by a force sensor. The reinforcement learning agent is trained inside this simulator. The trained model is directly applied to the real fin actuator. (Real2Sim2Real)
• Figure 3: Experimental setup: ROS2 computer sends motor commands to the fin motor driver, that generates movement and produces forces that are measured by the force sensor.
• Figure 4: Fin actuator and force sensor and axes from top and side view.
• Figure 5: Full architecture of surrogate model: (A) PosNet: two convolutional layers followed by three linear layers. (B) ForceNet: LSTM layer followed by two linear layers, dropout and final ouput linear layer.