Obstacle-Aware Navigation of Soft Growing Robots via Deep Reinforcement Learning

TL;DR

The paper addresses navigation of soft growing vine-like robots through cluttered environments by coupling a planar constant-curvature kinematic model with an obstacle-interaction, strain-energy-based formulation. A Deep Q-Network maps states $\in \mathbb{R}^9$ to discrete actuation pairs $[\dot{s},\dot{\kappa}]$, trained with an $\epsilon$-greedy policy to reach a specified goal while avoiding obstacles. Through simulations of fixed-goal, varying-goal, and obstacle-aware scenarios, the approach demonstrates the robot's ability to exploit obstacles and adapt its shape, achieving efficient goal-reaching and highlighting both the promise and the need for future work on continuous-action controls. This work advances soft robotics by integrating plant-inspired growth with learning-based navigation for operation in real-world, constrained environments.

Abstract

Soft growing robots, are a type of robots that are designed to move and adapt to their environment in a similar way to how plants grow and move with potential applications where they could be used to navigate through tight spaces, dangerous terrain, and hard-to-reach areas. This research explores the application of deep reinforcement Q-learning algorithm for facilitating the navigation of the soft growing robots in cluttered environments. The proposed algorithm utilizes the flexibility of the soft robot to adapt and incorporate the interaction between the robot and the environment into the decision-making process. Results from simulations show that the proposed algorithm improves the soft robot's ability to navigate effectively and efficiently in confined spaces. This study presents a promising approach to addressing the challenges faced by growing robots in particular and soft robots general in planning obstacle-aware paths in real-world scenarios.

Figures (13)

• Figure 1: Enhancement of Movement Skills in the Vine-Growing Robot Through Deep Reinforcement Learning: This process involves the robot's adaptive learning from its interactions with the environment.
• Figure 2: Operational Mechanics of the Growing Vine Robot. (a) The application of air pressure to the robot's central tube aids in extending the tip, as shown in (b). The steering mechanism, depicted in (c), is achieved by altering the air pressure in one or more of the soft pneumatic artificial muscles (sPAMs) surrounding the vine robot.
• Figure 3: Schematic of vine-like growing robot and its configuration parameters. The robot is characterized by its length $s$, curvature $\kappa = \frac{\theta}{s}$, and angle of robot plane $\phi$.
• Figure 4: Effect of a point obstacle on the continuum robot shape. On the left the robot has no interaction with the obstacle and the robot curvature is the same to all segments. On the right, the robot has collided with a point obstacle making the robot curvature not consistent along it's backbone.
• Figure 5: Illustration of the DQN Agent Architecture: This diagram depicts the agent as it observes state $s$ and executes action $a$. The process of transforming the observed state into an actionable decision is facilitated through a feed-forward neural network, which serves as the core mechanism for mapping observations to corresponding actions.