Uniform Convergence Beyond Glivenko-Cantelli

TL;DR

The paper broadens uniform convergence beyond the classical $P$-Glivenko–Cantelli framework by introducing $UME$-learnability, which asks when a collection of distributions on $\\{0,1\\}^\\mathbb{N}$ admits uniform mean estimation by any estimator. It shows that separability of the mean vectors is a sufficient condition for $UME$-learnability, yet provides non-separable examples (notably a tree-structured family) that are still learnable, highlighting fundamentally different techniques from separability-based arguments. It also proves that $UME$-learnability is closed under countable unions, resolving a conjecture in prior work and extending the result beyond two families. The work thus broadens the toolkit for uniform estimation in infinite-dimensional settings and suggests practical routes for replacing empirical risk minimization with $UME$-based estimators in learning problems, while leaving open the precise necessary-and-sufficient conditions for general non-separable cases.

Abstract

We characterize conditions under which collections of distributions on $\{0,1\}^\mathbb{N}$ admit uniform estimation of their mean. Prior work from Vapnik and Chervonenkis (1971) has focused on uniform convergence using the empirical mean estimator, leading to the principle known as $P-$ Glivenko-Cantelli. We extend this framework by moving beyond the empirical mean estimator and introducing Uniform Mean Estimability, also called $UME-$ learnability, which captures when a collection permits uniform mean estimation by any arbitrary estimator. We work on the space created by the mean vectors of the collection of distributions. For each distribution, the mean vector records the expected value in each coordinate. We show that separability of the mean vectors is a sufficient condition for $UME-$ learnability. However, we show that separability of the mean vectors is not necessary for $UME-$ learnability by constructing a collection of distributions whose mean vectors are non-separable yet $UME-$ learnable using techniques fundamentally different from those used in our separability-based analysis. Finally, we establish that countable unions of $UME-$ learnable collections are also $UME-$ learnable, solving a conjecture posed in Cohen et al. (2025).

Figures (1)

• Figure 1: Labeling for tree of depth $2$

Theorems & Definitions (14)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Lemma 5
• Theorem 6
• Theorem 7
• Proposition 8
• Proposition 9
• Proposition 10