RISE: Single Static Radar-based Indoor Scene Understanding

TL;DR

RISE tackles privacy-preserving indoor scene understanding with a single static mmWave radar by converting multipath ghosts into geometric cues. The core pipeline combines Bi-Angular Multipath Enhancement to reveal off-diagonal ghosts, multipath inversion to recover reflector geometry, and a Sim2Real Hierarchical Diffusion model to produce complete wall layouts and furniture detections from partial radar data. The authors create the RISE-Indoor Benchmark (50k frames, 100 trajectories across 11 environments) and demonstrate state-of-the-art layout accuracy (Chamfer ~16 cm) and first-mmWave furniture detection (IoU ~58%). This work lays a foundation for geometry-aware, privacy-preserving indoor perception with inexpensive, passive sensing and points to future extensions for dynamic or multi-person scenes in real-world deployments.

Abstract

Robust and privacy-preserving indoor scene understanding remains a fundamental open problem. While optical sensors such as RGB and LiDAR offer high spatial fidelity, they suffer from severe occlusions and introduce privacy risks in indoor environments. In contrast, millimeter-wave (mmWave) radar preserves privacy and penetrates obstacles, but its inherently low spatial resolution makes reliable geometric reasoning difficult. We introduce RISE, the first benchmark and system for single-static-radar indoor scene understanding, jointly targeting layout reconstruction and object detection. RISE is built upon the key insight that multipath reflections, traditionally treated as noise, encode rich geometric cues. To exploit this, we propose a Bi-Angular Multipath Enhancement that explicitly models Angle-of-Arrival and Angle-of-Departure to recover secondary (ghost) reflections and reveal invisible structures. On top of these enhanced observations, a simulation-to-reality Hierarchical Diffusion framework transforms fragmented radar responses into complete layout reconstruction and object detection. Our benchmark contains 50,000 frames collected across 100 real indoor trajectories, forming the first large-scale dataset dedicated to radar-based indoor scene understanding. Extensive experiments show that RISE reduces the Chamfer Distance by 60% (down to 16 cm) compared to the state of the art in layout reconstruction, and delivers the first mmWave-based object detection, achieving 58% IoU. These results establish RISE as a new foundation for geometry-aware and privacy-preserving indoor scene understanding using a single static radar.

Figures (11)

• Figure 1: RISE is the first single-radar system for object-level indoor scene understanding. Under standard millimeter-wave (mmWave) sensing (a.1), the radar directly observes only a small portion of the environment, producing sparse visible regions (a.2) due to strong specular reflections. To overcome this limitation, RISE performs multipath inversion(b.1) to exploit indirect reflections and recover surfaces that are not directly visible. Combined with our geometry-aware reconstruction pipeline, RISE enables accurate and complete indoor layout reconstruction and object detection (b.2).
• Figure 2: Multipath-induced Ghosts: Figure (a) illustrates the scenario in which ghost points appear. Figure (b) presents the corresponding XY heatmap generated by the radar.
• Figure 3: Pipeline. Given a sequence of mmWave signals, RISE first applies the Bi-Angular Multipath Enhancement (BAME) module to each frame to recover previously suppressed ghost paths. These enhanced observations are then processed by our multipath inversion module to estimate the underlying reflector geometry for every frame. By aggregating reflector estimates across the entire trajectory, RISE forms an initial, geometry-aware layout hypothesis. This coarse reconstruction is subsequently refined by our Sim2Real Hierarchical Diffusion (SRHD) model, which transforms fragmented radar observations into complete wall layouts and object masks.
• Figure 4: Multipth Inversion. Geometric relationships between ghost targets and reflectors.
• Figure 5: Bi-Angular Multipath Enhancement (BAME). Figure (a) shows a case where the ghost target cannot be recovered using conventional beamforming. Figure (b) illustrates the corresponding AOA--AOD response at the range of the ghost targets $G'_1$ and $G_1$. Notably, $G'_1$ disappears in the standard range--angle map (Fig. (a.2)) because its AOA and AOD are not aligned with the diagonal, causing it to be suppressed by the $\text{AOA}=\text{AOD}$ assumption in traditional processing.