Asymptotic non-linear shrinkage and eigenvector overlap for weighted sample covariance

TL;DR

This work extends Ledoit and Péché's asymptotic non-linear shrinkage to weighted, exponentially weighted covariance settings by deriving the limiting behavior of functionals Θ^g and the associated fixed-point transform X(z). It provides tractable oracle shrinkage formulas for rotation-invariant covariance and precision estimators under a weighted high-dimensional regime, together with explicit expressions for EWMA weights and a practical two-stage algorithm (recover H from data, then compute X and m) for implementation. The authors validate the theory with numerical experiments showing PRIAL gains and robustness to heavy tails, and establish a rigorous proof framework for the limiting objects, their continuity properties, and eigenvector overlaps. This yields principled, non-linear shrinkage rules applicable to time-varying or non-stationary data and offers a pathway to scalable, rotation-invariant covariance filtering in high dimensions.

Abstract

We compute asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators for weighted sample covariances, and the joint sample-population eigenvector overlap distribution, in the spirit of Ledoit and Péché. We detail explicitly the formulas for exponentially-weighted sample covariances. We propose an algorithm to numerically compute those formulas. Experimentally, we show the performance of the asymptotic non-linear shrinkage estimators. Finally, we test the robustness of the theory to a heavy-tailed distributions.

Figures (13)

• Figure 1: Shrinkage intensities $\lambda/h(\lambda)$ in function of $\lambda$, for $c=0.2$, $H = \frac{1}{5} \mathbf{1}_{[1,\infty[} + \frac{2}{5} \mathbf{1}_{[3,\infty[} + \frac{2}{5} \mathbf{1}_{[10,\infty[}$ and $D$ exponentially-weighted with parameter $\alpha \in \{0,2,5\}$ as introduced in Example \ref{['exEWMA']}.
• Figure 2: Heatmap of the bivariate overlap density $(\ell,t) \mapsto \phi(\ell,t)$ for $H$ uniform in $[5,6]$, concentration ratio $c=0.2$ and $D$ exponentially-weighted with parameter $\alpha=5$.
• Figure 3: (Top) $\lambda_{PCA}$ and $\lambda_{naive}$ in function of $\kappa$, for $H$ uniform in $[1,10]$, concentration ratio $c=0.1$ (left) and $c=0.5$ (right) and $D$ exponentially-weighted with parameter $\alpha=5$. (Down) The proportion of variance explained by the PCA in function of $\kappa$ for $\lambda_{PCA}$ (equal to $\kappa$ by construction) and $\lambda_{naive}$, for $H$ uniform in $[1,10]$, concentration ratio $c=0.1$ (left) and $c=0.5$ (right) and $D$ exponentially-weighted with parameter $\alpha \in \{0,3,5,8\}$.
• Figure 4: (Top) Histograms of sample eigenvalues, estimated population eigenvalues $\hat{H}$, and true population eigenvalues $H$. (Down) Estimated and true sample density computed on $S_F$ and sample eigenvalues' histogram.
• Figure 5: PRIAL of the oracle weighted shrunk estimator in function of the number of samples $N$, for $c=0.5$, with $H = \frac{1}{5} \mathbf{1}_{[1,\infty[} + \frac{2}{5} \mathbf{1}_{[3,\infty[} + \frac{2}{5} \mathbf{1}_{[10,\infty[}$ and exponentially-weighted distribution of parameter $\alpha$, Monte-Carlo of $n_{MC} = 50$ draws for each point.

Theorems & Definitions (80)

• Example 1: Standard weighting
• Example 2: Exponentially weighted scheme
• Remark 1
• Theorem 2.1: $F^{B_n}$ convergence, from Theorem 1.2.1 Zhang2007
• Definition 1: $\Theta^g_n$
• Theorem 2.2
• Theorem 2.3: Continuity on the real line
• Definition 2: Class of rotation-invariant estimator
• Proposition 1: Oracle rotation-invariant covariance estimator, Ledoit2009 p.9
• Definition 3: $\Delta_n$, from Ledoit2009