LiDAS: Lighting-driven Dynamic Active Sensing for Nighttime Perception

TL;DR

LiDAS introduces a closed-loop, perception-driven illumination framework that converts HD headlights into active vision actuators. By training a differentiable relighting operator with task-driven supervision on daytime-trained detectors and segmenters, it learns where to illuminate to maximize zero-shot performance while reducing energy use. Across synthetic and real-world driving scenarios, LiDAS delivers substantial gains in object detection and semantic segmentation with significant energy savings, and it complements domain-generalization methods like SoMA. The approach offers a practical, bolt-on solution for robust nighttime perception using hardware already on modern vehicles.

Abstract

Nighttime environments pose significant challenges for camera-based perception, as existing methods passively rely on the scene lighting. We introduce Lighting-driven Dynamic Active Sensing (LiDAS), a closed-loop active illumination system that combines off-the-shelf visual perception models with high-definition headlights. Rather than uniformly brightening the scene, LiDAS dynamically predicts an optimal illumination field that maximizes downstream perception performance, i.e., decreasing light on empty areas to reallocate it on object regions. LiDAS enables zero-shot nighttime generalization of daytime-trained models through adaptive illumination control. Trained on synthetic data and deployed zero-shot in real-world closed-loop driving scenarios, LiDAS enables +18.7% mAP50 and +5.0% mIoU over standard low-beam at equal power. It maintains performances while reducing energy use by 40%. LiDAS complements domain-generalization methods, further strengthening robustness without retraining. By turning readily available headlights into active vision actuators, LiDAS offers a cost-effective solution to robust nighttime perception.

Figures (9)

• Figure 1: Lighting‑driven Dynamic Active Sensing (LiDAS) is a bolt‑on active‑illumination system that converts high‑definition headlights into a vision actuator, projecting light where it most aids perception networks. It improves frozen, daytime‑trained object detection and semantic segmentation models while reducing power consumption.
• Figure 2: Training and deployment. LiDAS learns an illumination policy for active night vision. During training (top), the model predicts a light field $M_t$, which our differentiable relighting operator uses to synthesize the relit image $\hat{I}_t$. Frozen, daytime‑trained downstream heads provide task losses that supervise the policy. We unroll several refinement steps to mirror the closed‑loop setting. At deployment (bottom), the camera observes the lit scene, LiDAS adapts the light field $M_t$, and the HD headlight projects it back onto the scene, improving perception on the next camera frame. At runtime, any models can serve as the downstream heads.
• Figure 3: Qualitative results. We show YOLO11L (detection) and Mask2Former (semantic segmentation) predictions under different illumination policies. Only LiDAS$^{[1]}$ detects the left‑hand pedestrian and its segmentation map has well‑defined objects and no sky artifacts. LiDAS$^{[1]}$ lights objects of interest while leveraging ambient illumination, reducing power over the other car’s headlights (blue polygon) and the pedestrian’s white coat. Static shows that LiDAS typically reduces near-field light avoiding self-glare and reallocates power toward long‑range targets. Method$^{[x]}$ denotes power relative to LB.
• Figure 4: Budget‑aware illumination. LiDAS learns to prioritize illumination under tight power budgets, focusing only on the most informative regions. As the budget grows, it progressively broadens coverage and begins to accentuate finer details that aid global scene understanding. Method$^{[x]}$ denotes power relative to LB.
• Figure 5: Qualitative results from the closed-loop evaluation on our car-mounted prototype. We compare YOLOv8L-Worldv2 predictions under classic LB/HB and our LiDAS illumination policy at several energy budgets. LiDAS detects all present objects even at reduced power. It makes the pedestrian fully visible and reduces self‑glare on the white vehicle, improving contrast. Method$^{[x]}$ denotes power relative to LB.