Hybrid Human-Machine Perception via Adaptive LiDAR for Advanced Driver Assistance Systems

TL;DR

The paper addresses improving ADAS LiDAR perception by integrating real-time driver gaze to guide sensing. It introduces a gaze-aware LiDAR that defines the driver's region of focus RoF_driver and concentrates range and resolution enhancements in the complementary RoI_LiDAR, while maintaining a fixed average power P via the constraint (Δ_Driver P_low + Δ_LiDAR P_high)/(2π) = P. The contributions include a concrete system design, a CARLA-based proof-of-concept with gaze data, and evaluations showing that range-control improves detection timing in fog and that combining range and resolution control yields higher point densities, implying safer and more robust perception. The work demonstrates the feasibility and potential safety benefits of hybrid human-machine sensing for ADAS, particularly under adverse weather conditions. This approach suggests a path toward more reliable perception by leveraging human attention to guide machine sensing without increasing overall energy consumption or scan cadence.

Abstract

Accurate environmental perception is critical for advanced driver assistance systems (ADAS). Light detection and ranging (LiDAR) systems play a crucial role in ADAS; they can reliably detect obstacles and help ensure traffic safety. Existing research on LiDAR sensing has demonstrated that adapting the LiDAR's resolution and range based on environmental characteristics can improve machine perception. However, current adaptive LiDAR approaches for ADAS have not explored the possibility of combining the perception abilities of the vehicle and the human driver, which can potentially further enhance the detection performance. In this paper, we propose a novel system that adapts LiDAR characteristics to human driver's visual perception to enhance LiDAR sensing outside human's field of view. We develop a proof-of-concept prototype of the system in the virtual environment CARLA. Our system integrates real-time data on the driver's gaze to identify regions in the environment that the driver is monitoring. This allows the system to optimize LiDAR resources by dynamically increasing the LiDAR's range and resolution in peripheral areas that the driver may not be attending to. Our simulations show that this gaze-aware LiDAR enhances detection performance compared to a baseline standalone LiDAR, particularly in challenging environmental conditions like fog. Our hybrid human-machine sensing approach potentially offers improved safety and situational awareness in real-time driving scenarios for ADAS applications.

Figures (4)

• Figure 1: Overview of our proposed system. An implementation of the proposed system that determines the driver's region of focus $RoF_{\text{driver}}$, the region of interest for the LiDAR, from a shifted visual acuity function. The amount of shift is equal to the gaze direction ($\theta_{g}$). The proposed system achieves a higher range and a higher resolution outside $RoF_{\text{driver}}$ than a comparable standard LiDAR.
• Figure 2: The simulation scenario used in our evaluation. The ego vehicle (black vehicle) stands at a T-cross intersection and two other vehicles approach from the left and from the right.
• Figure 3: Time-to-arrival (TTA) comparison (in seconds) for four evaluated LiDAR control methods across three fog levels.
• Figure 4: Density of LiDAR data points in the LiDAR's region of interest for four evaluated LiDAR control methods across three fog levels.