Beyond Visibility Limits: A DRL-Based Navigation Strategy for Unexpected Obstacles

TL;DR

The paper tackles safety limitations of distance-based DRL navigation in dynamic environments by introducing DRL-NSUO, which incorporates a LiDAR-derived rate of environmental change into the reward and uses curriculum learning to progressively emphasize environmental stability. The approach leverages a SAC-based policy with LiDAR preprocessing and a reciprocal distance transform to heighten sensitivity to nearby obstacles, resulting in safer and more reliable navigation without reliance on global planners. Key contributions include the environmental-change reward design with curriculum scheduling, a real-time LiDAR change-rate metric $v_c^{(t)}$, and extensive validation on the BARN dataset showing higher success rates and lower collision rates, plus real-world demonstrations. This work provides a practical, safe local-planning solution for human-robot collaboration and logistics applications in crowded and dynamic environments.

Abstract

Distance-based reward mechanisms in deep reinforcement learning (DRL) navigation systems suffer from critical safety limitations in dynamic environments, frequently resulting in collisions when visibility is restricted. We propose DRL-NSUO, a novel navigation strategy for unexpected obstacles that leverages the rate of change in LiDAR data as a dynamic environmental perception element. Our approach incorporates a composite reward function with environmental change rate constraints and dynamically adjusted weights through curriculum learning, enabling robots to autonomously balance between path efficiency and safety maximization. We enhance sensitivity to nearby obstacles by implementing short-range feature preprocessing of LiDAR data. Experimental results demonstrate that this method significantly improves both robot and pedestrian safety in complex scenarios compared to traditional DRL-based methods. When evaluated on the BARN navigation dataset, our method achieved superior performance with success rates of 94.0% at 0.5 m/s and 91.0% at 1.0 m/s, outperforming conservative obstacle expansion strategies. These results validate DRL-NSUO's enhanced practicality and safety for human-robot collaborative environments, including intelligent logistics applications.

Figures (11)

• Figure 1: Illustration of the goal-driven robot navigation problem.
• Figure 2: The diagram demonstrates the distance-based robot's behavior of maintaining proximity to the wall during its traversal. In real-world test, it may result in collisions or pedestrian injuries, as shown in Fig. 10b and Fig. 10d.
• Figure 3: Neural Network architectures used for training.
• Figure 4: Simulated training environment Env1.
• Figure 5: A comparison chart of the LiDAR data change rate in the environment for DRL-NSUO and IPAPRec (Distance-based DRL algorithm) when passing through the same corner map Env2 (with the same starting point, endpoint, speed, and acceleration).