TOPGN: Real-time Transparent Obstacle Detection using Lidar Point Cloud Intensity for Autonomous Robot Navigation

TL;DR

TOPGN addresses real-time detection and safe navigation around transparent obstacles by leveraging lidar point-cloud intensities organized into a three-layer 2D grid map $I^t_{3L}$. It identifies transparent-obstacle neighborhoods (TONs) within $\,\mathcal{G}^t$, linearly extrapolates obstacle extents via tangent lines to form $\,\mathcal{G}^t_{extrap}$, and integrates this information into a navigation cost map $I^t_{nav}$ to drive collision avoidance and mapping. The method achieves real-time performance ($\sim$50 Hz on CPU) and demonstrates superior accuracy (e.g., at least $12.74\%$ higher $F_1$ and up to $38.46\%$ MAE reduction) and navigation success (at least 2x better) across indoor and outdoor scenarios with varying lighting, materials, and shapes, compared against RGB-based and existing lidar-based glass-detection approaches. TOPGN's approach remains robust to lighting changes and reflections, enabling safe navigation in unmapped environments and providing a practical framework for real-time transparent-obstacle perception and environment mapping; limitations include lidar blind spots and dependence on sufficient vertical resolution, with future work exploring low-FOV sensors and higher-order extrapolation.

Abstract

We present TOPGN, a novel method for real-time transparent obstacle detection for robot navigation in unknown environments. We use a multi-layer 2D grid map representation obtained by summing the intensities of lidar point clouds that lie in multiple non-overlapping height intervals. We isolate a neighborhood of points reflected from transparent obstacles by comparing the intensities in the different 2D grid map layers. Using the neighborhood, we linearly extrapolate the transparent obstacle by computing a tangential line segment and use it to perform safe, real-time collision avoidance. Finally, we also demonstrate our transparent object isolation's applicability to mapping an environment. We demonstrate that our approach detects transparent objects made of various materials (glass, acrylic, PVC), arbitrary shapes, colors, and textures in a variety of real-world indoor and outdoor scenarios with varying lighting conditions. We compare our method with other glass/transparent object detection methods that use RGB images, 2D laser scans, etc. in these benchmark scenarios. We demonstrate superior detection accuracy in terms of F-score improvement at least by 12.74% and 38.46% decrease in mean absolute error (MAE), improved navigation success rates (at least two times better than the second-best), and a real-time inference rate (~50Hz on a mobile CPU). We will release our code and challenging benchmarks for future evaluations upon publication.

Figures (11)

• Figure 1: Our method can robustly detect transparent obstacles in scenes with varying illumination in real-time ($\sim 50$ Hz). The figure shows the robot's trajectories in one trial when a planner fox1997dwa used our method (in green), GDNet gdnet (in blue), Translab translab (in red), Glass-SLAM glass-slam (in yellow), and 2D laser scans (in black) to detect transparent obstacles in unknown environments. RGB segmentation methods gdnettranslab are affected by strong lighting changes, and motion blur in these environments, causing collisions with glass, or freezing during navigation. SLAM methods such as glass-slam require $\sim 3$ seconds to update the locally sensed obstacles on to a map, leading to collisions. Our method's accurate detection of transparent and opaque obstacles facilitates safe, collision-free navigation in unmapped, unknown environments.
• Figure 2: The Gaussian distribution of point cloud intensities observed when the robot faces obstacles with different levels of transparency and shapes. The average intensities along with the distribution depend on the transparency (high transparency leads to lower average intensity), and shape (high curvature leads to lower average reflected intensity). The corresponding intensity maps at three different height ranges defined by equation \ref{['eqn:3-layers']} are shown, and the peak intensity neighborhood reflected from the glass appears in $I^t_{mid}$ due to our definitions of its range, and is highlighted by the yellow circle. Our formulation detects this pattern and linearly extrapolates the transparent obstacle's shape from it to safely navigate unknown environments. The green parallelograms show regions of interest which are defined by equation \ref{['eqn:roi_definition']}.
• Figure 3: (a) The green lines depict the light ray that is incident at $0^{\circ}$ on a transparent obstacle (in blue). The line segment perpendicular to this light ray is represented in red and is extended from the point of incidence on either side by the radius of the robot $r_{rob}$. (b) The same scenario on $\mathcal{G}^t$ where the light ray connects the robot (in yellow) with $(r^j_{cen}, c^j_{cen})$. The red lines are extended on either side by $r_{rob}/s$ grids in red. All grids on the other side of the red lines are considered as obstacles for time instant $t$. A few of the robot's instantaneous candidate trajectories when integrated with a planner fox1997dwa are shown in blue, yellow, and pink. An optimal trajectory is chosen based on its distance away from obstacles, and the progress/heading towards the goal. In this scenario, the pink trajectory is preferred over the others as it is away from obstacles.
• Figure 4: TOPGN's overall system architecture. Three-layered intensity map $I^t_{3L}$ is extracted from the 3D lidar point cloud to isolate the transparent obstacles. The linear extrapolation is performed on isolated obstacles to estimate the true shape of the obstacles for collision avoidance. The extrapolated transparent obstacles are combined with the navigation cost map $I^t_{nav}$ to generate collision-free and goal-reaching actions from the planner. This overall framework demonstrates superior real-time transparent obstacle detection capabilities compared to state-of-the-art vision-based and lidar-based approaches.
• Figure 5: $I^t_{mid}$ for the curved glass scenario shown in Fig. \ref{['fig:gaussian']} [center], and the transformed $\mathcal{G}^{t-3}$ (blue square), $\mathcal{G}^{t-6}$ (yellow square), and $\mathcal{G}^{t-9}$ (green square) added to it. The centers of these squares depict the robot's movement as time progresses. This addition reconstructs the true shape of the transparent obstacle for mapping.

Theorems & Definitions (2)

• Lemma IV.1
• proof