Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Bounds for Tyler's M-Estimator for Elliptical Distributions

Lap Chi Lau, Akshay Ramachandran

TL;DR

This work establishes optimal finite-sample and algorithmic guarantees for Tyler's M-estimator of the shape matrix under Elliptical distributions. By connecting Tyler's estimator to frame scaling and introducing the novel ∞-expansion pseudorandom condition, the authors prove that random frames satisfy the necessary expansion at the Gaussian-like optimal threshold n ~ d, yielding tight relative operator norm error bounds and ensuring linear convergence of Tyler's iterative procedure at n ~ d. The results close the gap with Gaussian covariance estimation and extend beyond to provide constructive scaling analyses that may apply to related frame theory problems and tensor normal models. Overall, the paper advances both statistical guarantees and algorithmic understanding of robust shape estimation in heavy-tailed elliptical models, with practical implications for high-dimensional covariance-like estimation beyond Gaussian assumptions.

Abstract

A fundamental problem in statistics is estimating the shape matrix of an Elliptical distribution. This generalizes the familiar problem of Gaussian covariance estimation, for which the sample covariance achieves optimal estimation error. For Elliptical distributions, Tyler proposed a natural M-estimator and showed strong statistical properties in the asymptotic regime, independent of the underlying distribution. Numerical experiments show that this estimator performs very well, and that Tyler's iterative procedure converges quickly to the estimator. Franks and Moitra recently provided the first distribution-free error bounds in the finite sample setting, as well as the first rigorous convergence analysis of Tyler's iterative procedure. However, their results exceed the sample complexity of the Gaussian setting by a $\log^{2} d$ factor. We close this gap by proving optimal sample threshold and error bounds for Tyler's M-estimator for all Elliptical distributions, fully matching the Gaussian result. Moreover, we recover the algorithmic convergence even at this lower sample threshold. Our approach builds on the operator scaling connection of Franks and Moitra by introducing a novel pseudorandom condition, which we call $\infty$-expansion. We show that Elliptical distributions satisfy $\infty$-expansion at the optimal sample threshold, and then prove a novel scaling result for inputs satisfying this condition.

Optimal Bounds for Tyler's M-Estimator for Elliptical Distributions

TL;DR

This work establishes optimal finite-sample and algorithmic guarantees for Tyler's M-estimator of the shape matrix under Elliptical distributions. By connecting Tyler's estimator to frame scaling and introducing the novel ∞-expansion pseudorandom condition, the authors prove that random frames satisfy the necessary expansion at the Gaussian-like optimal threshold n ~ d, yielding tight relative operator norm error bounds and ensuring linear convergence of Tyler's iterative procedure at n ~ d. The results close the gap with Gaussian covariance estimation and extend beyond to provide constructive scaling analyses that may apply to related frame theory problems and tensor normal models. Overall, the paper advances both statistical guarantees and algorithmic understanding of robust shape estimation in heavy-tailed elliptical models, with practical implications for high-dimensional covariance-like estimation beyond Gaussian assumptions.

Abstract

A fundamental problem in statistics is estimating the shape matrix of an Elliptical distribution. This generalizes the familiar problem of Gaussian covariance estimation, for which the sample covariance achieves optimal estimation error. For Elliptical distributions, Tyler proposed a natural M-estimator and showed strong statistical properties in the asymptotic regime, independent of the underlying distribution. Numerical experiments show that this estimator performs very well, and that Tyler's iterative procedure converges quickly to the estimator. Franks and Moitra recently provided the first distribution-free error bounds in the finite sample setting, as well as the first rigorous convergence analysis of Tyler's iterative procedure. However, their results exceed the sample complexity of the Gaussian setting by a factor. We close this gap by proving optimal sample threshold and error bounds for Tyler's M-estimator for all Elliptical distributions, fully matching the Gaussian result. Moreover, we recover the algorithmic convergence even at this lower sample threshold. Our approach builds on the operator scaling connection of Franks and Moitra by introducing a novel pseudorandom condition, which we call -expansion. We show that Elliptical distributions satisfy -expansion at the optimal sample threshold, and then prove a novel scaling result for inputs satisfying this condition.
Paper Structure (19 sections, 28 theorems, 92 equations)

This paper contains 19 sections, 28 theorems, 92 equations.

Table of Contents

  1. Introduction
  2. Our Results
  3. Techniques
  4. Preliminaries and Technical Overview
  5. Elliptical Distributions and Tyler's M-Estimator
  6. Frame Scaling
  7. Technical Overview
  8. Organization
  9. Optimal Sample Complexity via $\infty$-Expansion
  10. Frame Scaling with $\infty$-Expansion
  11. $\infty$-Expansion for Random Frames
  12. Convergence of Tyler's Iterative Procedure
  13. Conclusion
  14. Bibliography
  15. Proof of \ref{['t:frameMainInfty']}
  16. ...and 4 more sections

Key Result

Theorem 1.1

Given $n \gtrsim \frac{d}{\varepsilon^{2}}$ samples from an elliptical distribution with shape $\Sigma \in \operatorname{PD}(d)$, where $\varepsilon$ is at most a small constant, Tyler's M-Estimator $\hat{\Sigma}$ satisfies with probability $\geq 1 - \exp( - \Omega(\varepsilon^{2} n) )$.

Theorems & Definitions (52)

  • Theorem 1.1
  • Theorem 1.2
  • Definition 2.1: Elliptical Distribution
  • Definition 2.2: Tyler's M-Estimator
  • Definition 2.3
  • Definition 2.4
  • Definition 2.6: Frame Scaling Problem
  • Lemma 2.7: Example 2.3 in FM20
  • Lemma 2.8: Observation 3.1 in FM20
  • Definition 2.9: Quantum Expansion
  • ...and 42 more