Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices

Prathapasinghe Dharmawansa, Saman Atapattu, Jamie Evans, Merouane Debbah

TL;DR

This work tackles detecting a non-random signal in colored noise with an unknown covariance by using Roy's largest root, the leading eigenvalue of the whitened sample covariance. The authors derive a novel exact finite-dimensional c.d.f. for the leading eigenvalue of a complex non-central $F$-matrix with a rank-one non-centrality parameter using an orthogonal-polynomial approach, enabling precise ROC analysis in both finite and high-dimensional regimes. They show that the detector exhibits CFAR behavior under $\mathcal{H}_0$ and reveal phase-transition phenomena: in certain scaling regimes the SNR must grow as $O(p)$ or $O(p^2)$ to maintain detection power, with a sharp power drop in sub-critical high-dimensional settings. The results have practical implications for detection in MIMO radar and similar systems where training data are limited and dimensionality is large, providing actionable insights into how many signal-bearing samples $p$ and the SNR must scale to achieve reliable detection.

Abstract

This paper investigates the signal detection problem in colored noise with an unknown covariance matrix. In particular, we focus on detecting a non-random signal by capitalizing on the leading eigenvalue (a.k.a. Roy's largest root) of the whitened sample covariance matrix as the test statistic. To this end, the whitened sample covariance matrix is constructed via \(m\)-dimensional \(p \) plausible signal-bearing samples and \(m\)-dimensional \(n \) noise-only samples. Since the signal is non-random, the whitened sample covariance matrix turns out to have a {\it non-central} \(F\)-distribution with a rank-one non-centrality parameter. Therefore, the performance of the test entails the statistical characterization of the leading eigenvalue of the non-central \(F\)-matrix, which we address by deriving its cumulative distribution function (c.d.f.) in closed-form by leveraging the powerful orthogonal polynomial approach in random matrix theory. This new c.d.f. has been instrumental in analyzing the receiver operating characteristic (ROC) of the detector. We also extend our analysis into the high dimensional regime in which \(m,n\), and \(p\) diverge such that \(m/n\) and \(m/p\) remain fixed. It turns out that, when \(m=n\) and fixed, the power of the test improves if the signal-to-noise ratio (SNR) is of at least \(O(p)\), whereas the corresponding SNR in the high dimensional regime is of at least \(O(p^2)\). Nevertheless, more intriguingly, for \(m<n\) with the SNR of order \(O(p)\), the leading eigenvalue does not have power to detect {\it weak} signals in the high dimensional regime.

Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices

TL;DR

This work tackles detecting a non-random signal in colored noise with an unknown covariance by using Roy's largest root, the leading eigenvalue of the whitened sample covariance. The authors derive a novel exact finite-dimensional c.d.f. for the leading eigenvalue of a complex non-central -matrix with a rank-one non-centrality parameter using an orthogonal-polynomial approach, enabling precise ROC analysis in both finite and high-dimensional regimes. They show that the detector exhibits CFAR behavior under and reveal phase-transition phenomena: in certain scaling regimes the SNR must grow as or to maintain detection power, with a sharp power drop in sub-critical high-dimensional settings. The results have practical implications for detection in MIMO radar and similar systems where training data are limited and dimensionality is large, providing actionable insights into how many signal-bearing samples and the SNR must scale to achieve reliable detection.

Abstract

This paper investigates the signal detection problem in colored noise with an unknown covariance matrix. In particular, we focus on detecting a non-random signal by capitalizing on the leading eigenvalue (a.k.a. Roy's largest root) of the whitened sample covariance matrix as the test statistic. To this end, the whitened sample covariance matrix is constructed via -dimensional plausible signal-bearing samples and -dimensional noise-only samples. Since the signal is non-random, the whitened sample covariance matrix turns out to have a {\it non-central} -distribution with a rank-one non-centrality parameter. Therefore, the performance of the test entails the statistical characterization of the leading eigenvalue of the non-central -matrix, which we address by deriving its cumulative distribution function (c.d.f.) in closed-form by leveraging the powerful orthogonal polynomial approach in random matrix theory. This new c.d.f. has been instrumental in analyzing the receiver operating characteristic (ROC) of the detector. We also extend our analysis into the high dimensional regime in which , and diverge such that and remain fixed. It turns out that, when and fixed, the power of the test improves if the signal-to-noise ratio (SNR) is of at least \(O(p)\), whereas the corresponding SNR in the high dimensional regime is of at least \(O(p^2)\). Nevertheless, more intriguingly, for with the SNR of order \(O(p)\), the leading eigenvalue does not have power to detect {\it weak} signals in the high dimensional regime.

Paper Structure

This paper contains 13 sections, 8 theorems, 121 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Detection Problem formulation
  3. C.D.F. of the Maximum Eigenvalue
  4. Preliminaries
  5. C.D.F. of the Leading Eigenvalue
  6. ROC of the Leading Eigenvalue Based Test
  7. Finite Dimensional Analysis
  8. High Dimensional Analysis
  9. Conclusion
  10. Proof of Theorem \ref{['thmain']}
  11. An Alternative Proof of Corollary \ref{['corzeroalpha']}
  12. Proof of Proposition \ref{['pmatint']}
  13. The evaluation of $\mathcal{Q}_m(t,y)$

Key Result

Theorem 1

If $\boldsymbol{R}\sim\mathcal{CW}_m\left(p,\boldsymbol{\Sigma},\boldsymbol{\Theta}\right)$ and $\boldsymbol{S}\sim\mathcal{CW}_m\left(n,\boldsymbol{\Sigma}\right)$ are independently distributed with $p,n\geq m$, then $\boldsymbol{F}=\boldsymbol{S}^{-1/2}\boldsymbol{R}\boldsymbol{S}^{-1/2}$ follows where $\boldsymbol{\Omega}=\boldsymbol{\Sigma}^{-1/2}\boldsymbol{MM}^H\boldsymbol{\Sigma}^{-1/2}$ d

Figures (8)

  • Figure 1: Comparison between the theoretical c.d.f. in Theorem \ref{['thmain']} with simulated values for various system configurations with $m=5$ and $\omega=2$.
  • Figure 2: The effect of $\omega$ on the CDF for $m=10, n=12$, and $p=15$. The red dashed curve corresponds to Corollary \ref{['corzeromean']}.
  • Figure 3: The effect of $p$ on the c.d.f. of scaled random variable $\lambda_{\max}/p$ corresponding to the configuration $m=n$ with $\omega=O(1)$ and $\omega=O(p)$.
  • Figure 4: Limiting distributions of the scaled maximum eigenvalue $\lambda_{\max}/m^2$ as $m,n,p\to\infty$ such that $m=n$ and $m/p\to 1/2$ with $\omega=O(p)$ and $\omega=O(p^2)$.
  • Figure 5: The effect of $n,p$ and $\omega$ on ROC profile for $m=4$.
  • ...and 3 more figures

Theorems & Definitions (13)

  • Definition 1
  • Definition 2
  • Remark 1
  • Theorem 1
  • Theorem 2
  • Definition 3
  • Definition 4
  • Corollary 1
  • Theorem 3
  • Corollary 2
  • ...and 3 more