Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Depth and Image Fusion for Road Obstacle Detection Using Stereo Camera

Oleg Perezyabov, Mikhail Gavrilenkov, Ilya Afanasyev

TL;DR

This work tackles the challenge of detecting road obstacles when their appearance is unknown and lighting varies, a scenario where purely ML/DL approaches may underperform. It proposes a depth-and-image fusion pipeline that combines stereo-derived depth information with RGB-based object analysis, leveraging SLIC superpixels, ground-plane estimation via RANSAC, and DBSCAN clustering, followed by a fusion stage using non-maximum suppression. The key contributions include a dual-channel obstacle detection framework, an enhanced monocular RGB processing approach with targeted pre-processing, and a demonstration in underground parking conditions showing robust detection and tracking of small objects. The results indicate that integrating depth with RGB cues improves detection accuracy and robustness, with practical implications for parking safety and intelligent transport monitoring in challenging environments.

Abstract

This paper is devoted to the detection of objects on a road, performed with a combination of two methods based on both the use of depth information and video analysis of data from a stereo camera. Since neither the time of the appearance of an object on the road, nor its size and shape is known in advance, ML/DL-based approaches are not applicable. The task becomes more complicated due to variations in artificial illumination, inhomogeneous road surface texture, and unknown character and features of the object. To solve this problem we developed the depth and image fusion method that complements a search of small contrast objects by RGB-based method, and obstacle detection by stereo image-based approach with SLIC superpixel segmentation. We conducted experiments with static and low speed obstacles in an underground parking lot and demonstrated the successful work of the developed technique for detecting and even tracking small objects, which can be parking infrastructure objects, things left on the road, wheels, dropped boxes, etc.

Depth and Image Fusion for Road Obstacle Detection Using Stereo Camera

TL;DR

This work tackles the challenge of detecting road obstacles when their appearance is unknown and lighting varies, a scenario where purely ML/DL approaches may underperform. It proposes a depth-and-image fusion pipeline that combines stereo-derived depth information with RGB-based object analysis, leveraging SLIC superpixels, ground-plane estimation via RANSAC, and DBSCAN clustering, followed by a fusion stage using non-maximum suppression. The key contributions include a dual-channel obstacle detection framework, an enhanced monocular RGB processing approach with targeted pre-processing, and a demonstration in underground parking conditions showing robust detection and tracking of small objects. The results indicate that integrating depth with RGB cues improves detection accuracy and robustness, with practical implications for parking safety and intelligent transport monitoring in challenging environments.

Abstract

This paper is devoted to the detection of objects on a road, performed with a combination of two methods based on both the use of depth information and video analysis of data from a stereo camera. Since neither the time of the appearance of an object on the road, nor its size and shape is known in advance, ML/DL-based approaches are not applicable. The task becomes more complicated due to variations in artificial illumination, inhomogeneous road surface texture, and unknown character and features of the object. To solve this problem we developed the depth and image fusion method that complements a search of small contrast objects by RGB-based method, and obstacle detection by stereo image-based approach with SLIC superpixel segmentation. We conducted experiments with static and low speed obstacles in an underground parking lot and demonstrated the successful work of the developed technique for detecting and even tracking small objects, which can be parking infrastructure objects, things left on the road, wheels, dropped boxes, etc.
Paper Structure (10 sections, 3 equations, 15 figures, 2 tables)

This paper contains 10 sections, 3 equations, 15 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology and Algorithm Implementation
  4. RGB image-based obstacle detection
  5. Stereo-based obstacle detection
  6. Fusion
  7. Experimental Setup, Tests and Results
  8. Conclusions and Discussion
  9. Conclusions
  10. Discussion

Figures (15)

  • Figure 1: The methodology of depth and image fusion for road obstacle detection using stereo camera
  • Figure 2: Original test image (left) and its graph segmentation with implemented color map (right) without pre-processing
  • Figure 3: Original test image (left) and its graph segmentation visualized as the colormap (right) using Pegtop soft light blending with an inverse copy of the image
  • Figure 4: Original images (left) and images with the results of their graph segmentation (right). The bottom images are before pre-processing, and the top image - after Pegtop soft light blending (with its inverse copy) applied to the bottom original image
  • Figure 5: Original test image after RGB to HSV color space conversion (left) and its graph segmentation visualized as the colormap (right)
  • ...and 10 more figures