DART: Implicit Doppler Tomography for Radar Novel View Synthesis

TL;DR

DART tackles radar novel view synthesis by learning an implicit tomographic map from range-Doppler radar measurements. By adopting a NeRF-inspired framework that outputs a base reflectivity $\bar{\sigma}$ and base transmittance $\bar{\alpha}$ together with angular encodings via spherical harmonics, DART renders accurate range-Doppler images from novel poses through a physics-informed Doppler-cone rendering model. The approach is trained end-to-end with a differentiable forward renderer and evaluated on a novel handheld radar dataset with lidar-based pose localization, outperforming lidar-based simulators, CFAR-based baselines, and nearest-neighbor methods in SSIM and visual fidelity, while enabling tomographic mapping of material properties. These results demonstrate the feasibility of data-driven, physics-aware radar scene modeling and open avenues for radar-specific localization, mapping, and multi-modal sensing; however, the method relies on static scenes and accurate velocity estimates, suggesting future work to relax motion constraints and enable real-time single-chip radar solutions.

Abstract

Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.

Figures (22)

• Figure 1: DART uses scans from a handheld radar to learn an implicit tomography of a scene in order to accurately render scans from novel viewpoints (left). DART's implicit tomography can also be sampled to map the radar properties of a scene (right).
• Figure 2: NeRF's pinhole camera model renders a pixel (left) by integrating along a ray (right, green), while DART's range-Doppler model renders a pixel (middle) by integrating along a velocity-dependent (right, blue) circle (right, red).
• Figure 3: DART tackles radar novel view synthesis by learning a neural implicit map of the world from a trajectory of radar measurements. We make key radar-specific decisions in choosing (1) a high quality radar representation space --- Range-Doppler, (2) a world model that captures radar interactions --- $\sigma$ and $\alpha$ with spherical harmonics coefficients, (3) a network architecture to model our desired representation --- Instant NGP, and (4) an optimized radar rendering and training methodology --- Range-Doppler specific rendering.
• Figure 4: Doppler arises due to differences in relative velocities between points with different relative angles to the radar (left). Each range value (red) corresponds to a sphere, while each Doppler value corresponds to a cone (green). The intersection forms the range-Doppler pixel (see Fig. \ref{['fig:toy_rangedoppler']}).
• Figure 5: Example (validation) range-Doppler frames and descriptive photos of our method and baselines. DART accurately reproduces the overall radar image, though it lacks the resolution to resolve smaller weak reflectors. Lidar can model weak reflectors, but cannot accurately scale them due to a lack of radar-specific information, while Nearest produces radar-realistic but inaccurate images since exhaustively measuring all possible poses is impractical. Finally, CFAR cannot model transmittance or measure the "volume" of a point.