Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Metric Entropy and Minimax Risk of Ellipsoids with an Application to Pinsker's Theorem

Thomas Allard

TL;DR

This work develops a unified framework to quantify ellipsoid compactness in Hilbert spaces through type-$\tau$ integrals $I_{\tau}$ and the semi-axis counting function $M_{\mu}$. It proves that metric entropy is governed by $I_{1}$ and the minimax risk by $I_{2}$ and $I_{3}$, enabling a precise connection between averaged and pointwise decay of ellipsoid axes. The authors establish a general bias–variance decomposition for minimax risk across three regimes (via a parameter $b$ from RC) and apply the results to Sobolev ellipsoids, yielding refined Pinsker-type theorems on general domains and with second-order terms. The Sobolev ellipsoid analysis leverages Weyl’s law and Ivrii’s two-term expansion to obtain sharp entropy and risk constants, linking Laplacian spectral data to ellipsoidal geometry and advancing classical results in metric entropy and minimax theory.

Abstract

We study how large an $\ell^2$ ellipsoid is by introducing type-$τ$ integrals that capture the average decay of its semi-axes. These integrals turn out to be closely related to standard complexity measures: we show that the metric entropy of the ellipsoid is asymptotically equivalent to the type-1 integral, and that the minimax risk in non-parametric estimation is asymptotically determined by the type-2 and type-3 integrals. This allows us to retrieve and sharpen classical results about metric entropy and minimax risk of ellipsoids through a systematic analysis of the type-$τ$ integrals, and yields an explicit formula linking the two. As an application, we improve on the best-known characterization of the metric entropy of the Sobolev ellipsoid, and extend Pinsker's Sobolev theorem in two ways: (i) to any bounded open domain in arbitrary finite dimension, and (ii) by providing the second-order term in the asymptotic expansion of the minimax risk.

Metric Entropy and Minimax Risk of Ellipsoids with an Application to Pinsker's Theorem

TL;DR

This work develops a unified framework to quantify ellipsoid compactness in Hilbert spaces through type- integrals and the semi-axis counting function . It proves that metric entropy is governed by and the minimax risk by and , enabling a precise connection between averaged and pointwise decay of ellipsoid axes. The authors establish a general bias–variance decomposition for minimax risk across three regimes (via a parameter from RC) and apply the results to Sobolev ellipsoids, yielding refined Pinsker-type theorems on general domains and with second-order terms. The Sobolev ellipsoid analysis leverages Weyl’s law and Ivrii’s two-term expansion to obtain sharp entropy and risk constants, linking Laplacian spectral data to ellipsoidal geometry and advancing classical results in metric entropy and minimax theory.

Abstract

We study how large an ellipsoid is by introducing type- integrals that capture the average decay of its semi-axes. These integrals turn out to be closely related to standard complexity measures: we show that the metric entropy of the ellipsoid is asymptotically equivalent to the type-1 integral, and that the minimax risk in non-parametric estimation is asymptotically determined by the type-2 and type-3 integrals. This allows us to retrieve and sharpen classical results about metric entropy and minimax risk of ellipsoids through a systematic analysis of the type- integrals, and yields an explicit formula linking the two. As an application, we improve on the best-known characterization of the metric entropy of the Sobolev ellipsoid, and extend Pinsker's Sobolev theorem in two ways: (i) to any bounded open domain in arbitrary finite dimension, and (ii) by providing the second-order term in the asymptotic expansion of the minimax risk.
Paper Structure (26 sections, 13 theorems, 326 equations)

This paper contains 26 sections, 13 theorems, 326 equations.

Table of Contents

  1. Introduction
  2. Notation and terminology.
  3. Metric Entropy and Type-1 Integral
  4. Minimax Risk, Type-2, and Type-3 Integrals
  5. Metric Entropy and Minimax Risk
  6. Sobolev Ellipsoids and Pinsker's Theorem
  7. Typical Asymptotics
  8. Proofs of Main Results
  9. Proof of Theorem \ref{['thm:metric-entropy-integral-form']}
  10. A series formula for $I_{1}$.
  11. A lower bound on metric entropy.
  12. Preparation.
  13. Two upper bounds on metric entropy.
  14. Proof of \ref{['eq:partial-result-OM']}.
  15. Proof of \ref{['eq:upper-bound-integral-ME2-OM']}.
  16. ...and 11 more sections

Key Result

Theorem 2

Let $\mu=\{\mu_n\}_{n\in\mathbb{N}^*}$ be a sequence such that $\mu_n\to 0$ as $n\to\infty$, with corresponding semi-axis-counting function $M_{\mu}$ satisfying (RC). Then, the metric entropy of $\mathcal{E}_{\mu}$ and its type-$1$ integral $I_{1}$ satisfy In particular, we have

Theorems & Definitions (28)

  • Definition 1
  • Theorem 2
  • proof : Proof sketch.
  • Definition 3
  • Theorem 4
  • proof : Proof sketch
  • Theorem 5
  • proof : Proof sketch
  • Lemma 6
  • proof : Proof sketch
  • ...and 18 more