Real-Time Obstacle Avoidance for a Mobile Robot Using CNN-Based Sensor Fusion

TL;DR

<3-5 sentence high-level summary> The paper tackles real-time obstacle avoidance for indoor mobile robots by comparing three end-to-end CNNs that fuse RGB and depth data from a RealSense D415. NetConEmb delivers the best robustness and accuracy, particularly in known and unknown environments, while NetEmb offers significant parameter reduction with comparable performance. Ablation experiments and sensor-failure analyses highlight the value of depth information and RGB-D fusion for reliable navigation. Real-time onboard deployment on a Jetson Nano demonstrates practical viability for vision-based autonomous navigation in diverse indoor settings.

Abstract

Obstacle avoidance is a critical component of the navigation stack required for mobile robots to operate effectively in complex and unknown environments. In this research, three end-to-end Convolutional Neural Networks (CNNs) were trained and evaluated offline and deployed on a differential-drive mobile robot for real-time obstacle avoidance to generate low-level steering commands from synchronized color and depth images acquired by an Intel RealSense D415 RGB-D camera in diverse environments. Offline evaluation showed that the NetConEmb model achieved the best performance with a notably low MedAE of $0.58 \times 10^{-3}$ rad/s. In comparison, the lighter NetEmb architecture, which reduces the number of trainable parameters by approximately 25\% and converges faster, produced comparable results with an RMSE of $21.68 \times 10^{-3}$ rad/s, close to the $21.42 \times 10^{-3}$ rad/s obtained by NetConEmb. Real-time navigation further confirmed NetConEmb's robustness, achieving a 100\% success rate in both known and unknown environments, while NetEmb and NetGated succeeded only in navigating the known environment.

Figures (9)

• Figure 1: End-to-end navigation approach.
• Figure 2: RQT graph generated during online navigation.
• Figure 3: Geometrical representation of the differential drive mobile robot showing coordinate frame definitions.
• Figure 4: Training and validation loss comparison of the three CNN architectures with early stopping applied, demonstrating convergence characteristics and relative performance.
• Figure 5: NetEmb architecture showing the late-fusion approach with embedded feature concatenation, resulting in reduced parameter count while maintaining performance characteristics.