Range and Angle Estimation with Spiking Neural Resonators for FMCW Radar

TL;DR

This work tackles the high data bandwidth and energy demands of automotive FMCW radar by introducing spiking neural resonators based on the resonate-and-fire model to perform real-time range-angle estimation. By combining dendritic vector multiplication for angle, neural resonators for range, and gradient-driven spiking, the approach enables continuous processing with spike-based readout, eliminating the need to store full data frames. The method achieves state-of-the-art or comparable detection accuracy to Fourier-based processing while reducing data transmission to as little as 0.02% of float-32 FT and lowering latency through online processing. The results on simulated and real radar data demonstrate substantial noise reduction and robust multi-target detection, highlighting the potential of neuromorphic radar sensors for low-power, high-speed perception in autonomous systems.

Abstract

Automotive radar systems face the challenge of managing high sampling rates and large data bandwidth while complying with stringent real-time and energy efficiency requirements. The growing complexity of autonomous vehicles further intensifies these requirements. Neuromorphic computing offers promising solutions because of its inherent energy efficiency and parallel processing capacity. This research presents a novel spiking neuron model for signal processing of frequency-modulated continuous wave (FMCW) radars that outperforms the state-of-the-art spectrum analysis algorithms in latency and data bandwidth. These spiking neural resonators are based on the resonate-and-fire neuron model and optimized to dynamically process raw radar data while simultaneously emitting an output in the form of spikes. We designed the first neuromorphic neural network consisting of these spiking neural resonators that estimates range and angle from FMCW radar data. We evaluated the range-angle maps on simulated datasets covering multiple scenarios and compared the results with a state-of-the-art pipeline for radar processing. The proposed neuron model significantly reduces the processing latency compared to traditional frequency analysis algorithms, such as the Fourier transformation (FT), which needs to sample and store entire data frames before processing. The evaluations demonstrate that these spiking neural resonators achieve state-of-the-art detection accuracy while emitting spikes simultaneously to processing and transmitting only 0.02 % of the data compared to a float-32 FT. The results showcase the potential for neuromorphic signal processing for FMCW radar systems and pave the way for designing neuromorphic radar sensors.

Figures (5)

• Figure 1: Geometrical visualization of the radar signals. On the left, antenna layout with $N_\text{vx}$ virtual antennas in one line and a spacing $b$ between consecutive antennas. Transmitted and reflected signals are indicated with arrows, and the direction-of-arrival (DoA) is given as angle $\theta$. In the middle, schematic view of temporal dynamics of the IF signals $x_m(t)$ with frequency $\omega$, where $m$ is the antenna index. Frequency analysis along the temporal dimension provides information on the range of an object. On the right, schematic view of the complex value $\exp(i m \phi)$ over virtual antennas. Frequency analysis along the antenna dimension provides information on the DoA.
• Figure 2: Network of spiking neural resonators. Reflected signals are indicated as arrows, detected by virtual antennas. Each virtual antenna passes its IF signal $x_m(t)$ to a neural resonator with given eigenfrequency $\omega_j$ weighted by the complex weight vector $\vec{w}_l$. Along the vertical dimension, complex weight vectors optimized for a specific angle $\theta$ are visualized (gray, black). Along the horizontal dimension, the eigenfrequency of the neuron changes optimized for a specific range $r$. The transmitted spikes can be visualized as range-angle map.
• Figure 3: Comparison of neuron dynamics and spiking functions. The left column shows the neuron behavior when a target is present, whereas in the right column no target is present. The first row shows the the neuron's magnitude $\|s\|$, and its estimated maximum $s_\text{max}$ and the estimated maximum width $w_\text{max}$ between the magnitude and its maximum. The estimated envelope $\Lambda$ (grey) follows a lower boundary of the magnitude $\|s\|$. The second row shows the behaviour of the adaptive threshold spiking function. In the top row, positive spikes are indicated black and negative spikes red. A positive (negative) spike is generated when $s_\text{max}$ ($w_\text{max}$) reaches the threshold $u_\text{th}^{s}$ ($u_\text{th}^{w}$). The negative width $-w_\text{max}$ is shown for better visibility. The third row shows the behaviour of the rate-coded LIF spiking function. A present target results in a spike rate, whereas no target does not produce any spike. The fourth row shows the behaviour of the time-coded LIF spiking function. A present target results in a single spike, whereas no target does not produce any spike.
• Figure 4: Early detection evaluation for adaptive threshold (adapt.), time-coded (time), rate-coded (rate), and the gradient (grad) model. After every $64$ data sample, we determine the neural resonator network's F-score, precision, and recall. The maximum value of each model normalizes the values. For comparison, a linear reference is shown in red. The F-Score of the adaptive threshold, rate-coded, and gradient model increases almost linearly, as the resolution of the Fourier transform also increases linearly with the number of samples. The time-coded model shows a different behavior, as most informative spikes happen at the end of a chirp.
• Figure 5: Comparison of range-angle maps from FMCW radar sensor data of the RADICAL dataset for one scene. (Left) RBG image of the RADICAL dataset of the corresponding scene. (Right, Top) Full-resolution range-angle maps showing the full spectrum. (Right, Bottom) Zoomed-in extract of the two people detected by the radar sensor. The noise level of the range-angle map of the Fourier Transform is higher than the non-spiking and spiking range-angle map of neural resonator networks. The gradient and adaptive threshold spiking function maps are similar, as are the rate-coded and time-coded maps.