MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception

TL;DR

MMVR addresses the scarcity of indoor radar datasets by introducing a large-scale, multi-view millimeter-wave radar dataset for indoor perception. It collects 345K frames from 25 subjects across 6 rooms over 9 days, with two orthogonal high-resolution radar heatmaps and synchronized RGB-D data, enabling benchmarks for object detection, pose estimation, and instance segmentation. The authors adapt or re-implement baseline radar methods and establish two evaluation protocols (random and cross-environment splits), reporting results and ablations that reveal the impact of temporal framing, dual views, and backbone choices. MMVR provides a publicly available benchmark to advance robust indoor radar perception, with potential applications in robot navigation, energy management, and elderly care.

Abstract

Compared with an extensive list of automotive radar datasets that support autonomous driving, indoor radar datasets are scarce at a smaller scale in the format of low-resolution radar point clouds and usually under an open-space single-room setting. In this paper, we scale up indoor radar data collection using multi-view high-resolution radar heatmap in a multi-day, multi-room, and multi-subject setting, with an emphasis on the diversity of environment and subjects. Referred to as the millimeter-wave multi-view radar (MMVR) dataset, it consists of $345$K multi-view radar frames collected from $25$ human subjects over $6$ different rooms, $446$K annotated bounding boxes/segmentation instances, and $7.59$ million annotated keypoints to support three major perception tasks of object detection, pose estimation, and instance segmentation, respectively. For each task, we report performance benchmarks under two protocols: a single subject in an open space and multiple subjects in several cluttered rooms with two data splits: random split and cross-environment split over $395$ 1-min data segments. We anticipate that MMVR facilitates indoor radar perception development for indoor vehicle (robot/humanoid) navigation, building energy management, and elderly care for better efficiency, user experience, and safety. The MMVR dataset is available at https://doi.org/10.5281/zenodo.12611978.

Figures (23)

• Figure 1: Radar signal representations: pre-CFAR heatmap versus post-CFAR point cloud (PC) corresponding to a scene in our MMVR dataset. The heatmap ((a) for a 3D view and (b) for the top-down view) shows an extended vertical profile of a subject in terms of multiple clustered reflections over the elevation (height) domain. In contrast, the CFAR operation compares the heatmap with a threshold plane (the semi-transparent surface in (a)) to declare a few detection points (red squares in (a) and (c)), greatly suppressing weaker reflections from the subject.
• Figure 2: Snapshots of indoor radar heatmap datasets in Table \ref{['table: heatmap']}. HuPR (HuPR23) and HIBER (RFMask23) were collected at a single location and, respectively, multiple locations inside a single open-space/open-foreground room. Our MMVR was collected in multiple locations over multiple open-foreground (d1-d4) and cluttered (d5-d9) rooms.
• Figure 3: MMVR sensor setup (a) with two mmWave radars and one RGB-D camera; (b) each frame includes two orthogonal high-resolution radar heatmaps and the synchronized RGB image; (c): the two radar heatmaps are inputs to radar perception models for the three perception tasks with the supervision from RGB-based labels.
• Figure 4: Comparison between (a) low-resolution and (b) high-resolution radar heatmaps of a corner reflector.
• Figure 5: Camera-Radar coordinate calibration.