High-probability minimax lower bounds

TL;DR

This work develops a high-probability minimax framework by introducing the minimax quantile $\\mathcal{M}(\\delta)$ and its lower counterpart $\\mathcal{M}_-(\\delta)$ to capture tail behaviour of estimation losses. It systematically derives high-probability lower bounds via Le Cam and Fano-type methods and a reduction from local minimax risk to quantile bounds, then applies the machinery to diverse problems: robust and Gaussian mean estimation, covariance estimation, sparse linear regression, nonparametric density estimation, isotonic regression, and stochastic convex optimization. The results show that the minimax quantiles can be characterized up to universal constants and, in many cases, matched by δ-independent estimators, highlighting potential adaptation across quantile levels. Overall, minimax quantiles offer a finer-grained benchmark than classical minimax risk and enable robust, tail-aware performance guarantees under heavy tails and contamination with practical implications for estimator design and evaluation.

Abstract

The minimax risk is often considered as a gold standard against which we can compare specific statistical procedures. Nevertheless, as has been observed recently in robust and heavy-tailed estimation problems, the inherent reduction of the (random) loss to its expectation may entail a significant loss of information regarding its tail behaviour. In an attempt to avoid such a loss, we introduce the notion of a minimax quantile, and seek to articulate its dependence on the quantile level. To this end, we develop high-probability variants of the classical Le Cam and Fano methods, as well as a technique to convert local minimax risk lower bounds to lower bounds on minimax quantiles. To illustrate the power of our framework, we deploy our techniques on several examples, recovering recent results in robust mean estimation and stochastic convex optimisation, as well as obtaining several new results in covariance matrix estimation, sparse linear regression, nonparametric density estimation and isotonic regression. Our overall goal is to argue that minimax quantiles can provide a finer-grained understanding of the difficulty of statistical problems, and that, in wide generality, lower bounds on these quantities can be obtained via user-friendly tools.

Figures (3)

• Figure 1: Illustration of the construction in Proposition \ref{['prop:KDE']}. Left: The functions $g$ (solid) and $h$ (dot-dashed) from \ref{['Eq:gh']}. Right: The functions $f_0$ (solid) and $f_1$ (dot-dashed) from \ref{['Eq:f0f1']} and \ref{['Eq:f0f12']}.
• Figure 2: Illustration of the functions in the proof of Proposition \ref{['prop:SCO-rate']} with $f:\mathcal{X} \times \{-1,1\}$ given by $f(x,y) = \gamma|x+yR|$.
• Figure 3: Illustration of the construction of $\widehat{\theta}^{(1)}$ in the proof of Theorem \ref{['lemma:expectation-lb-to-high-prob-lb']}. If $x\in S(\widehat{\theta})$, then we set $\widehat{\theta}^{(1)}(x) \coloneqq \widehat{\theta}(x)$; otherwise we set $\widehat{\theta}^{(1)}(x) \coloneqq \theta^{(1)} \in \Theta_1$.

Theorems & Definitions (55)

• Definition 1: Minimax quantile
• Proposition 2
• Definition 3: Lower minimax quantile
• Theorem 4
• Lemma 5: High-probability Le Cam's two-point lemma
• Corollary 6
• proof
• Lemma 7
• Theorem 8
• Proposition 9