Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework

Kohio Deflesselle, Mélodie Daniel, Aly Magassouba, Miguel Aranda, Olivier Ly

TL;DR

This work tackles the problem of safe maneuvering for double-Ackermann-steering robots under non-holonomic constraints. It introduces a Deep Reinforcement Learning framework based on Soft Actor-Critic (SAC) with Hindsight Experience Replay (HER) and CrossQ, enabling obstacle avoidance without handcrafted trajectories. The state and action definitions, plus a sparse but safety-conscious reward, yield robust performance; in simulation the method reaches up to 97% success on seen targets and 95% on unseen targets, with improvements in path efficiency (SPL). The results suggest strong potential for real-world deployment, with future work aimed at validation in cluttered real environments.

Abstract

We present a deep reinforcement learning framework based on Soft Actor-Critic (SAC) for safe and precise maneuvering of double-Ackermann-steering mobile robots (DASMRs). Unlike holonomic or simpler non-holonomic robots such as differential-drive robots, DASMRs face strong kinematic constraints that make classical planners brittle in cluttered environments. Our framework leverages the Hindsight Experience Replay (HER) and the CrossQ overlay to encourage maneuvering efficiency while avoiding obstacles. Simulation results with a heavy four-wheel-steering rover show that the learned policy can robustly reach up to 97% of target positions while avoiding obstacles. Our framework does not rely on handcrafted trajectories or expert demonstrations.

Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework

TL;DR

This work tackles the problem of safe maneuvering for double-Ackermann-steering robots under non-holonomic constraints. It introduces a Deep Reinforcement Learning framework based on Soft Actor-Critic (SAC) with Hindsight Experience Replay (HER) and CrossQ, enabling obstacle avoidance without handcrafted trajectories. The state and action definitions, plus a sparse but safety-conscious reward, yield robust performance; in simulation the method reaches up to 97% success on seen targets and 95% on unseen targets, with improvements in path efficiency (SPL). The results suggest strong potential for real-world deployment, with future work aimed at validation in cluttered real environments.

Abstract

We present a deep reinforcement learning framework based on Soft Actor-Critic (SAC) for safe and precise maneuvering of double-Ackermann-steering mobile robots (DASMRs). Unlike holonomic or simpler non-holonomic robots such as differential-drive robots, DASMRs face strong kinematic constraints that make classical planners brittle in cluttered environments. Our framework leverages the Hindsight Experience Replay (HER) and the CrossQ overlay to encourage maneuvering efficiency while avoiding obstacles. Simulation results with a heavy four-wheel-steering rover show that the learned policy can robustly reach up to 97% of target positions while avoiding obstacles. Our framework does not rely on handcrafted trajectories or expert demonstrations.

Paper Structure

This paper contains 13 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Method
  4. DRL Background
  5. State Space
  6. State Space
  7. Action Space
  8. Reward Function
  9. Experimental Results
  10. Environment setup
  11. Training setup
  12. Preliminary simulation results
  13. Conclusion

Figures (3)

  • Figure 1: Maneuvering a DASMR with DRL is challenging, as it requires temporary reward loss (circled in purple), which makes classical approaches sub-optimal. The red dot indicates the desired goal, the blue sphere denotes the obstacle, and the orange curve illustrates a feasible trajectory that avoids the obstacle while reaching the goal.
  • Figure 2: DASMR rotating around an instantaneous center of rotation (ICR).
  • Figure 3: Each of the two rows shows a different maneuver executed by our agent in simulation. The red dot indicates the goal, while the blue sphere denotes the obstacle.