Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral Properties of Generalized Correlation Matrices

Florent Benaych-Georges, Tomas Espana

TL;DR

The paper introduces a generalized, antisymmetric $\phi$-based framework for covariance and correlation, unifying classical measures like Pearson's $\rho$ and Kendall's $\tau$, and examines their spectra in high dimensions under the null of uncorrelated variables. Using random matrix theory, it proves that the empirical spectral distributions of $\mathcal{COV}^{\phi}$ and $\mathrm{COR}^{\phi}$ converge to affine transformations of the Marčenko–Pastur law, with parameters determined by the limiting moments of $\phi$; specifically, $t=\alpha/\beta$ and the limit measures are $\mathbb{MP}_q^{\alpha,\beta-\alpha}$ and $\mathbb{MP}_q^{t,1-t}$. For $d\ge 3$, antisymmetry induces a collapse of the spectrum around $\beta$ and $1$, respectively, aligning with known special cases (MP and Bandeira–Kendall). The results provide a robust spectral criterion for detecting null independence in high dimensions and open avenues for using higher-order $d$-wise interactions to identify correlation structure. The work combines detailed probabilistic bounds, a key polynomial identity, and careful matrix-approximation arguments to establish these MP-type limits under broad moment conditions.

Abstract

We introduce a family of coefficients that generalize the notion of correlation and explore their properties in the large dimensional multivariate case, showing that in the null case of uncorrelated variables, the spectrum of generalized correlation matrices is distributed according to an affine transformation of the Marchenko-Pastur law.

Spectral Properties of Generalized Correlation Matrices

TL;DR

The paper introduces a generalized, antisymmetric -based framework for covariance and correlation, unifying classical measures like Pearson's and Kendall's , and examines their spectra in high dimensions under the null of uncorrelated variables. Using random matrix theory, it proves that the empirical spectral distributions of and converge to affine transformations of the Marčenko–Pastur law, with parameters determined by the limiting moments of ; specifically, and the limit measures are and . For , antisymmetry induces a collapse of the spectrum around and , respectively, aligning with known special cases (MP and Bandeira–Kendall). The results provide a robust spectral criterion for detecting null independence in high dimensions and open avenues for using higher-order -wise interactions to identify correlation structure. The work combines detailed probabilistic bounds, a key polynomial identity, and careful matrix-approximation arguments to establish these MP-type limits under broad moment conditions.

Abstract

We introduce a family of coefficients that generalize the notion of correlation and explore their properties in the large dimensional multivariate case, showing that in the null case of uncorrelated variables, the spectrum of generalized correlation matrices is distributed according to an affine transformation of the Marchenko-Pastur law.

Paper Structure

This paper contains 15 sections, 12 theorems, 75 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Main result
  3. Proofs
  4. Notation
  5. Proof of Theorem \ref{['maintheorem']}
  6. Proof of Lemma \ref{['cutofflemma']}
  7. Proof of Lemma \ref{['mainlemma1']}
  8. Proof of Lemma \ref{['mainlemma2']}
  9. Proof of Lemma \ref{['lemconcdiagmat']}
  10. Appendix
  11. Marčenko-Pastur distributions $\mathbb{MP}_q$ and $\mathbb{MP}_q^{\alpha, \beta}$
  12. A polynomial identity
  13. Approximation and Frobenius norm
  14. Variance in a sparse graphical model
  15. McDiarmid-Azuma-Hoeffding concentration inequality for bounded differences

Key Result

Theorem 2.1

Let $d\ge 2$ be an integer. For each $n$, let $p$ be a positive integer, be an array of i.i.d. random variables taking values in a measured space $\mathcal{S}$, and $\phi:\mathcal{S}^d\to\mathbb{R}$ be an antisymmetric measurable function (all, except $d$, implicitly depending on $n$). We introduce some independent copies of $\mathsf{X}_1(1)$ and the function (so that the random variables $\mat

Figures (1)

  • Figure 1: Histogram of the eigenvalues of $\mathcal{COR}^{\phi}$ when $d\!=\!2, \phi\!=\!\tanh, p\!=\!500$ and $q\!=\!0.5$, with i.i.d. standard Gaussian data. The superimposed red line is the density function of the Marčenko-Pastur law $\mathbb{MP}_q^{t, 1-t}$ with $t\approx0.9$ (defined in Section \ref{['subsec:defMP']}), as predicted by our Theorem \ref{['maintheorem']}

Theorems & Definitions (20)

  • Theorem 2.1
  • Remark 2.2
  • Remark 2.3
  • Remark 2.4
  • Remark 2.5
  • Remark 2.6
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • ...and 10 more