Staircase Localization for Autonomous Exploration in Urban Environments

TL;DR

The paper addresses the challenge of locating stairs for autonomous exploration in urban environments. It introduces a cascade pipeline combining YOLOv5‑based stair detection to generate an ROI, M‑LSD‑tiny line segment extraction within the ROI, and depth‑aided stair localization to estimate stair position $P_{stair}$, orientation via $q_{stair}$, and direction. Contributions include a modular, RGB‑D, generalizable framework with outlier rejection via RANSAC, integration with the Information Roadmap (IRM) for planning, and extensive experiments across indoor/outdoor stairs showing robustness to lighting, occlusions, and clutter. The approach requires only a single RGB‑D camera, can accommodate different robotic platforms, and supports updating components as new detectors become available, achieving accurate localization up to several meters with favorable angular precision.

Abstract

A staircase localization method is proposed for robots to explore urban environments autonomously. The proposed method employs a modular design in the form of a cascade pipeline consisting of three modules of stair detection, line segment detection, and stair localization modules. The stair detection module utilizes an object detection algorithm based on deep learning to generate a region of interest (ROI). From the ROI, line segment features are extracted using a deep line segment detection algorithm. The extracted line segments are used to localize a staircase in terms of position, orientation, and stair direction. The stair detection and localization are performed only with a single RGB-D camera. Each component of the proposed pipeline does not need to be designed particularly for staircases, which makes it easy to maintain the whole pipeline and replace each component with state-of-the-art deep learning detection techniques. The results of real-world experiments show that the proposed method can perform accurate stair detection and localization during autonomous exploration for various structured and unstructured upstairs and downstairs with shadows, dirt, and occlusions by artificial and natural objects.

Figures (10)

• Figure 1: In our autonomous urban exploration system, an RGB-D camera is used to detect and localize stairs. A vision-based deep learning approach is employed to detect the stairs and provide an ROI. Within the ROI, we use line segments and corresponding depth data to estimate the pose of the stairs. This allows the robot to perform real-time stair localization from multiple perspectives.
• Figure 2: Illustration of the projection of a three-dimensional point and unprojection of a pixel. The projection and unprojection are performed based on camera specification. In this study, the unprojection of a pixel is used to generate a point cloud, which is mainly used in the stair localization module. The projection of a three-dimensional point is used to match the detected line segments and corresponding points in a point cloud.
• Figure 3: Pipeline of the proposed method. The proposed method consists of i) stair detection, ii) stair line segment detection, and iii) stair localization (including stair angle estimator and stair pose estimator) modules. The RGB color image and point cloud from a single RGB-D camera are used and processed in each module. The resulting output of the proposed algorithm includes the position, orientation, and direction (upstairs and downstairs) of the detected staircase.
• Figure 4: Example outcomes for each component using our proposed method. (a) color (RGB) image (b) stair detection result with a bounding box (c) stair line segment detection result (red lines: detected line segments, blue lines: outlier removed) (d) stair localization result (purple arrow: localized staircase in terms of position and orientation)
• Figure 5: The comparison of stair line (segment) detection between the proposed method and a classical CV method (Canny edge detection + Hough transformation). Images with different exposures are used for a staircase with indistinct stair lines (the edges of the stair are relatively blunt). Each column corresponds to i) the original image, ii) the result of the proposed method, iii) Canny edge detection, and iv) Hough transformation for line detection. The classical CV method could not detect stair lines for the highest exposure case. On the other hand, the proposed method detects consistent lines against different exposures.