Metric Entropy of Ellipsoids in Banach Spaces: Techniques and Precise Asymptotics

TL;DR

This work develops a unified block-decomposition framework to compute the metric entropy of infinite-dimensional $p$-ellipsoids with polynomially decaying semi-axes, extending prior results for exponentially decaying axes. By combining finite-block entropy bounds with density arguments and introducing mixed-ellipsoid constructions, the authors obtain sharp leading-term constants for $H(\varepsilon; \mathcal E_p, \|\cdot\|_q)$ for arbitrary $p,q\in[1,\infty]$, and they also derive a refined second-order term for $p=q=2$ and an exact entropy description for $p=q=\infty$. The results yield precise asymptotics and compactness criteria across all parameter regimes, and they are applied to Besov and Sobolev function classes, clarifying how domain geometry affects entropy and enabling sharper bounds relevant to nonparametric estimation and neural-network approximation. Collectively, these contributions advance the understanding of metric entropy in infinite dimensions and provide practically meaningful guidance for function approximation and learning in high- or infinite-dimensional settings.

Abstract

We develop new techniques for computing the metric entropy of ellipsoids -- with polynomially decaying semi-axes -- in Banach spaces. Besides leading to a unified and comprehensive framework, these tools deliver numerous novel results as well as substantial improvements and generalizations of classical results. Specifically, we characterize the constant in the leading term in the asymptotic expansion of the metric entropy of $p$-ellipsoids with respect to $q$-norm, for arbitrary $p,q \in [1, \infty]$, to date known only in the case $p=q=2$. Moreover, for $p=q=2$, we improve upon classical results by specifying the second-order term in the asymptotic expansion. In the case $p=q=\infty$, we obtain a complete, as opposed to asymptotic, characterization of metric entropy and explicitly construct optimal coverings. To the best of our knowledge, this is the first exact characterization of the metric entropy of an infinite-dimensional body. Application of our general results to function classes yields an improvement of the asymptotic expansion of the metric entropy of unit balls in Sobolev spaces and identifies the dependency of the metric entropy of unit balls in Besov spaces on the domain of the functions in the class. Sharp results on the metric entropy of function classes find application, e.g., in machine learning, where they allow to specify the minimum required size of deep neural networks for function approximation, nonparametric regression, and classification over these function classes.

Figures (2)

• Figure 1: The two-dimensional ellipsoid $\mathcal{E}^2_2$ is automatically covered in the dimension associated with $\mu_2$ by lining up $\varepsilon$-balls on the horizontal axis when $\varepsilon$ is larger than $\mu_2$.
• Figure 2: The shaded area of the two-dimensional ellipsoid $\mathcal{E}_\infty^2$ (left picture) is not covered when balls of radius $\varepsilon \simeq \mu_2$ are lined up on the horizontal axis. This problem disappears when considering balls of radius $\varepsilon \simeq d^{(1/q-1/p)} \mu_d = \sqrt{2}\mu_2$ (right picture).

Theorems & Definitions (19)

• Definition 1: Infinite-dimensional ellipsoids
• Definition 2: Regularly varying sequence
• Definition 3: Covering numbers and metric entropy
• Definition 4: Finite-dimensional ellipsoids
• Definition 5: Block decomposition
• Lemma 6: Decomposition
• proof : Proof.
• Lemma 7: Volume estimates, wainwrightHighDimensionalStatistics2019
• Theorem 8
• proof : Proof.