Task-Driven Implicit Representations for Automated Design of LiDAR Systems

TL;DR

This paper tackles the problem of automated LiDAR system design under arbitrary constraints. It introduces representing LiDAR configurations in a continuous 6D design space $\mathcal{D}$ and learning task-specific implicit densities $p_{\text{task}}(\mathbf{d}; \Theta)$ with flow-based modeling, guided by a target density $p^*(\mathbf{d})$ that encodes surface proximity and ray visibility. It then synthesizes sensors by fitting parametric distributions $q(\mathbf{d}|\theta)$ to $p(\mathbf{d})$ via EM, enabling constraint-aware design. The approach is validated on face-scanning, tracking, and object-detection tasks, showing adaptive designs that handle occlusions and motion while reducing bandwidth, suggesting a path toward rapid, constraint-aware computational sensor design.

Abstract

Imaging system design is a complex, time-consuming, and largely manual process; LiDAR design, ubiquitous in mobile devices, autonomous vehicles, and aerial imaging platforms, adds further complexity through unique spatial and temporal sampling requirements. In this work, we propose a framework for automated, task-driven LiDAR system design under arbitrary constraints. To achieve this, we represent LiDAR configurations in a continuous six-dimensional design space and learn task-specific implicit densities in this space via flow-based generative modeling. We then synthesize new LiDAR systems by modeling sensors as parametric distributions in 6D space and fitting these distributions to our learned implicit density using expectation-maximization, enabling efficient, constraint-aware LiDAR system design. We validate our method on diverse tasks in 3D vision, enabling automated LiDAR system design across real-world-inspired applications in face scanning, robotic tracking, and object detection.

Figures (11)

• Figure 1: We propose a task-driven implicit representation for automated LiDAR system design. Our implicit representation can be used to automatically propose well-suited LiDAR systems for a broad range of 3D vision tasks, such as scanning, tracking, and object detection, and can incorporate realistic physical constraints for computational LiDAR system generation.
• Figure 2: Visualization of simplified (6D$\rightarrow$3D) LiDAR design space for a single 2D scene point. Scene points can be observed by infinitely many geometrically-equivalent LiDAR designs, which form density-weighted curves in our design space. We then further weight design points along curves by their ray visibility, incorporating scene occlusions.
• Figure 3: Existing LiDARs represented in design space. LiDARs form discrete sampling volumes in our 6D design space, reflecting sensor origin, motion, direction, FoV, and time gating.
• Figure 4: Visualization of implicit LiDAR density for 2D surface scenes in simplified (6D$\rightarrow$3D) design space. We visualize a learned implicit LiDAR density over the parameter domain $(x \in [0,1], \phi \in [\frac{\pi}{4},\frac{3\pi}{4}],\tau\in[0,1.2])$ for a class of 2D scenes. High-density points in this space, representing high-proximity and high-visibility measurements, can be used to guide LiDAR sensor design.
• Figure 5: Representing Sensors and Imposing Constraints in LiDAR Design Space. Sensors are represented as parametric distributions in 6D design space, and are learned via expectation-maximization (EM) over the task-driven implicit density. Physical or user-imposed constraints can be easily incorporated by restricting the design space support or by clamping the parameters of the learned sensor distributions.