MM-2FSK: Multimodal Frequency Shift Keying for Ultra-Efficient and Robust High-Resolution MIMO Radar Imaging

TL;DR

This work tackles fast, high-resolution 3D radar imaging of static and dynamic targets with mmWave MIMO systems by addressing the limitations of bandwidth and computation. It introduces MM-2FSK, a multimodal extension of the two-frequency FSK approach that leverages per-point depth priors from an optical depth sensor to enable rapid, high-precision reconstruction with only a few frequencies. The method triangulates depth priors from the optical depth map into radar coordinates, forms per-point candidates, and refines depths via residual phasors, achieving robust performance even with varying surface geometries. Evaluations on the MAROON dataset show MM-2FSK delivering millimeter-scale depth accuracy and competitive quality to backprojection with full bandwidth while dramatically reducing measurement and computation time, highlighting its potential for real-time multi-sensor radar imaging and tracking. $\Delta d_{\max} = \frac{c}{4\Delta f}$ and related phase-derived corrections are central to the depth estimation, enabling per-point updates $d = \tilde{d} + \Delta d$ in a robust, GPU-accelerated framework.

Abstract

Accurate reconstruction of static and rapidly moving targets demands three-dimensional imaging solutions with high temporal and spatial resolution. Radar sensors are a promising sensing modality because of their fast capture rates and their independence from lighting conditions. To achieve high spatial resolution, MIMO radars with large apertures are required. Yet, they are infrequently used for dynamic scenarios due to significant limitations in signal processing algorithms. These limitations impose substantial hardware constraints due to their computational intensity and reliance on large signal bandwidths, ultimately restricting the sensor's capture rate. One solution of previous work is to use few frequencies only, which enables faster capture and requires less computation; however, this requires coarse knowledge of the target's position and works in a limited depth range only. To address these challenges, we extend previous work into the multimodal domain with MM-2FSK, which leverages an assistive optical depth sensing modality to obtain a depth prior, enabling high framerate capture with only few frequencies. We evaluate our method using various target objects with known ground truth geometry that is spatially registered to real millimeter-wave MIMO radar measurements. Our method demonstrates superior performance in terms of depth quality, being able to compete with the time- and resource-intensive measurements with many frequencies.

Figures (5)

• Figure 1: In our work, we extend the 2FSK imaging principle to the multimodal domain (MM-2FSK). Given a unified scalar depth prior, $\widetilde{d}$, for each point, the 2FSK method iteratively refines the current estimate with a per-point depth correction factor, $\Delta d$, up to a limited extent, given by the maximum unambiguous depth correction, $d_{\text{max}}$. In contrast, our method receives per-point depth priors from a secondary depth sensor, without requiring knowledge about the target position, and is more robust towards targets of varying surface depth.
• Figure 2: Simplified visualization of the 2FSK depth correction process, where complex, analytic signals are schemed as periodic, real-valued sine waves. The 2FSK principle computes the depth correction $\Delta d$ based on the residual of the phase, $\Delta\varphi$, that remains after correlating the two single-frequency signals with a signal hypothesis, constructed with the depth prior $\widetilde{d}$. The top row depicts the two residual signals at frequencies $f_1$ and $f_2$, respectively. Using these, a complex differential signal with frequency $\Delta f$ is calculated, as depicted in the bottom row. This differential signal is used to adjust the current depth guess and is constrained by the maximum unambiguous depth correction, $\pm d_{\textit{max}}$. The correction factor is centered around the zero-crossing of the signal within the first period -- or here, half of the period, due to signal simplification -- pointing into the direction where the residual phase yields zero. Due to the $2\pi$-periodicity of the continuous signal, the residual phase corresponding to the ground truth depth, $d_{\text{max}}$ may lie within a different signal period, resulting in the depth correction not producing the intended outcome.
• Figure 3: Visualization of the depth prior generation: We first create a closed triangle mesh using Delaunay triangulation on 2D pixels corresponding to valid 3D point samples from the optical depth sensor; illustrated here in 2D as blue line sets. The mesh is then transformed into the radar's coordinate space and re-sampled via rasterization on the radar pixel grid to generate the candidate point set $\mathcal{P}$.
• Figure 4: MM-2FSK reconstructions for the Cardboard and Wood Ball objects, compared across different frequency configurations. The point clouds are color-coded based on the residual phasor magnitude, which ap-pro-xi-ma-te-ly corresponds to the intensity of the signal. Higher band-widths exhibit fewer artifacts as they are less sensible to noisy phase variations.
• Figure 5: Comparison of the reconstructed point clouds for backprojection, 2FSK, 3FSK, and MM-2FSK across different frequency configurations and views, with standard BP using the full 10 GHz frequency spectrum at 128 frequency steps. The radar point clouds are color-coded based on the residual phasor magnitude, approximately corresponding to the intensity of the signal. Side and top views overlay the reconstructed point cloud with the ground-truth point cloud for the Bottle object at 30 cm object-to-sensor distance. The MM-2FSK method exhibits fewer artifacts and is closer to the ground truth than backprojection and 2FSK at the same frequency configuration. Additionally, depending on the frequency difference, MM-2FSK performs as well as or better than 3FSK, which employs a greater number of frequencies.