MAER-Nav: Bidirectional Motion Learning Through Mirror-Augmented Experience Replay for Robot Navigation

TL;DR

MAER-Nav tackles the action-flexibility gap in DRL-based robot navigation by introducing Mirror-Augmented Experience Replay, which generates synthetic backward experiences from successful forward trajectories. Coupled with bidirectional action recovery and curriculum learning, MAER-Nav enables robust bidirectional motion learning using a dual-buffer replay system and a SAC-based network with a LiDAR-focused input transform. Extensive simulation and real-world experiments show MAER-Nav outperforms state-of-the-art DRL methods and a raw MAER baseline, achieving higher success rates, lower collisions, and improved backward maneuvering in confined and dynamic environments. The approach preserves forward navigation performance while expanding maneuverability, offering a practical path toward more versatile autonomous navigation without additional sensors or reward engineering.

Abstract

Deep Reinforcement Learning (DRL) based navigation methods have demonstrated promising results for mobile robots, but suffer from limited action flexibility in confined spaces. Conventional DRL approaches predominantly learn forward-motion policies, causing robots to become trapped in complex environments where backward maneuvers are necessary for recovery. This paper presents MAER-Nav (Mirror-Augmented Experience Replay for Robot Navigation), a novel framework that enables bidirectional motion learning without requiring explicit failure-driven hindsight experience replay or reward function modifications. Our approach integrates a mirror-augmented experience replay mechanism with curriculum learning to generate synthetic backward navigation experiences from successful trajectories. Experimental results in both simulation and real-world environments demonstrate that MAER-Nav significantly outperforms state-of-the-art methods while maintaining strong forward navigation capabilities. The framework effectively bridges the gap between the comprehensive action space utilization of traditional planning methods and the environmental adaptability of learning-based approaches, enabling robust navigation in scenarios where conventional DRL methods consistently fail.

Figures (8)

• Figure 1: Proposed MAER-Nav structure. By mirroring the states and actions of successful episodes in the mirror-augmented buffer, we aim to enable the local planner trained by the DRL-based method to learn the bidirectional capability.
• Figure 2: The overall network structure of MAER-Nav.
• Figure 3: Training environment in ROS StageStage_rosWiki.
• Figure 4: Four unseen test environments: Narrow Corner Corridor, Multiple Walls, SEnv1 and SEnv2 for simulation test.
• Figure 5: Trajectory comparison in corridor and wall scenarios: successful forward and backward navigation trajectories of MAER-Nav (black robot) versus the failed backward trajectories of DRL-DCLP (blue robot). 'S' and 'G' represent the robot's start point and goal point, respectively. Start, goal, crash and stuck positions, along with the robot's trajectories, are indicated in the legend.