Floor extraction and door detection for visually impaired guidance

TL;DR

This work addresses safe navigation in unknown environments for visually impaired users by a wearable perception system that fuses an RGB-D camera with a wide-field fish-eye camera. It introduces a two-stage floor recovery (depth-based floor plane extraction followed by color-based floor expansion) and a door-detection pipeline that uses line hypotheses and Cross-Ratio verification within the fused data, with extrinsic calibration between the cameras via Gauss-Newton optimization and Scaramuzza's omnidirectional model. The key contributions include a floor area expansion up to about $20\times$ the RGB-D field of view and a conservative door detector that minimizes false positives, validated across indoor and outdoor scenes. The results demonstrate robust real-time performance and potential for guiding users through safe trajectories, highlighting practical impact for visually impaired mobility and autonomous navigation.

Abstract

Finding obstacle-free paths in unknown environments is a big navigation issue for visually impaired people and autonomous robots. Previous works focus on obstacle avoidance, however they do not have a general view of the environment they are moving in. New devices based on computer vision systems can help impaired people to overcome the difficulties of navigating in unknown environments in safe conditions. In this work it is proposed a combination of sensors and algorithms that can lead to the building of a navigation system for visually impaired people. Based on traditional systems that use RGB-D cameras for obstacle avoidance, it is included and combined the information of a fish-eye camera, which will give a better understanding of the user's surroundings. The combination gives robustness and reliability to the system as well as a wide field of view that allows to obtain many information from the environment. This combination of sensors is inspired by human vision where the center of the retina (fovea) provides more accurate information than the periphery, where humans have a wider field of view. The proposed system is mounted on a wearable device that provides the obstacle-free zones of the scene, allowing the planning of trajectories for people guidance.

Figures (8)

• Figure 1: (a): Proposed combination of cameras; (b): View from a standard RGB-D camera; (c): Projection of the RGB-D field of view into the fish-eye image; (d): Floor detection from the combination of cameras proposed.
• Figure 2: (a): Proposed device mounted on the user; (b): Reference systems for the user and the camera.
• Figure 3: Stages on the floor detection. Starting from the RGB-D camera, where 3D information is obtained from point clouds to extract the floor plane, to the fish-eye image, where working with colour information in order to extend the floor plane in a wider range into the environment.
• Figure 4: (a): Fish-eye image; (b): RGB-D floor detection; (c): Floor expansion; (d): 3D reconstruction of the obstacle-free zones.
• Figure 5: (a): Fish-eye image; (b): Extended floor and Vertical line extraction; (c): Door hypotheses regions; (d): Door detection.