Latent-space metrics for Complex-Valued VAE out-of-distribution detection under radar clutter

TL;DR

The problem addressed is robust radar out-of-distribution detection under clutter, tackled with complex-valued VAEs trained under the complex-valued ELBO $L_{CVAE} = L_{rec} + \beta D_{KL}$. The paper compares three detectors—reconstruction-based CVAE_MSE, an empirical latent KL score, and a Hermitian Mahalanobis score—against the classical ANMF-Tyler detector across synthetic Gaussian and compound-Gaussian clutter as well as CSIR real data. Key findings show CVAE_MSE leading in synthetic scenarios, while ANMF-FP dominates under compound clutter and Mahalanobis improves Doppler-0 robustness on real data; KLD is consistently brittle. The results provide practical guidelines for selecting reconstruction, latent-space, or classical detectors depending on clutter realism and the Doppler cell, highlighting the value of modeling complex latent structure for radar OOD detection.

Abstract

We investigate complex-valued Variational AutoEncoders (CVAE) for radar Out-Of-Distribution (OOD) detection in complex radar environments. We proposed several detection metrics: the reconstruction error of CVAE (CVAE-MSE), the latent-based scores (Mahalanobis, Kullback-Leibler divergence (KLD)), and compared their performance against the classical ANMF-Tyler detector (ANMF-FP). The performance of all these detectors is analyzed on synthetic and experimental radar data, showing the advantages and the weaknesses of each detector.

Figures (3)

• Figure 1: Complex-Valued VAE network architecture
• Figure 2: Detection performance under different noise configurations, ($P_{fa}= 10^{-2}$).
• Figure 3: On the left: range-slow time map. On the right: Doppler-pulse spectrogram for range bin 5605.