Detection of Signals in Colored Noise: Leading Eigenvalue Test for Non-central $F$-matrices
Prathapasinghe Dharmawansa, Saman Atapattu, Jamie Evans, Kandeepan Sithamparanathan
TL;DR
This work tackles the problem of detecting a non-random signal in colored noise with unknown covariance by using Roy's largest root test on the whitened sample covariance. It delivers exact finite-dimensional c.d.f.s for the leading eigenvalue of non-central $F$-matrices with rank-one non-centrality, enabling precise ROC analysis for the detector. The authors then extend the results to high-dimensional regimes, showing a phase-transition phenomenon: strong signals ($\bar{\gamma}>\bar{\gamma}_{\text{p}}$) are reliably detectable via a Tracy-Widom-based statistic, while weak signals fall below the detectability threshold. The findings provide rigorous tools for signal detection in modern colored-noise environments and offer practitioners exact performance characterizations across both classical and high-dimensional settings.
Abstract
This paper investigates the signal detection problem in colored noise with an unknown covariance matrix. In particular, we focus on detecting an unknown non-random signal by capitalizing on the leading eigenvalue of the whitened sample covariance matrix as the test statistic (a.k.a. Roy's largest root test). Since the unknown signal is non-random, the whitened sample covariance matrix turns out to have a non-central $F$-distribution. This distribution assumes a singular or non-singular form depending on whether the number of observations $p\lessgtr$ the system dimensionality $m$. Therefore, we statistically characterize the leading eigenvalue of the singular and non-singular $F$-matrices by deriving their cumulative distribution functions (c.d.f.). Subsequently, they have been utilized in deriving the corresponding receiver operating characteristic (ROC) profiles. We also extend our analysis into the high dimensional domain. It turns out that, when the signal is sufficiently strong, the maximum eigenvalue can reliably detect it in this regime. Nevertheless, weak signals cannot be detected in the high dimensional regime with the leading eigenvalue.
