Random sections of ellipsoids and the power of random information
Aicke Hinrichs, David Krieg, Erich Novak, Joscha Prochno, Mario Ullrich
TL;DR
It is proved that random information behaves very differently depending on whether $\sigma \in \ell_2$ or not, and the expected radius of random information tends to zero at least at rate $o(1/\sqrt{n})$ as $n\to\infty$.
Abstract
We study the circumradius of the intersection of an $m$-dimensional ellipsoid $\mathcal E$ with semi-axes $σ_1\geq\dots\geq σ_m$ with random subspaces of codimension $n$. We find that, under certain assumptions on $σ$, this random radius $\mathcal{R}_n=\mathcal{R}_n(σ)$ is of the same order as the minimal such radius $σ_{n+1}$ with high probability. In other situations $\mathcal{R}_n$ is close to the maximum $σ_1$. The random variable $\mathcal{R}_n$ naturally corresponds to the worst-case error of the best algorithm based on random information for $L_2$-approximation of functions from a compactly embedded Hilbert space $H$ with unit ball $\mathcal E$. In particular, $σ_k$ is the $k$th largest singular value of the embedding $H\hookrightarrow L_2$. In this formulation, one can also consider the case $m=\infty$, and we prove that random information behaves very differently depending on whether $σ\in \ell_2$ or not. For $σ\notin \ell_2$ random information is completely useless, i.e., $\mathbb E[\mathcal{R}_n] = σ_1$. For $σ\in \ell_2$ the expected radius of random information tends to zero at least at rate $o(1/\sqrt{n})$ as $n\to\infty$. In the important case $σ_k \asymp k^{-α} \ln^{-β}(k+1)$, where $α> 0$ and $β\in\mathbb R$, we obtain that $$ \mathbb E [\mathcal{R}_n(σ)] \asymp \begin{cases} σ_1 & : α<1/2 \,\text{ or }\, β\leqα=1/2 \\ σ_n \, \sqrt{\ln(n+1)} & : β>α=1/2 \\ σ_{n+1} & : α>1/2. \end{cases} $$ In the proofs we use a comparison result for Gaussian processes à la Gordon, exponential estimates for sums of chi-squared random variables, and estimates for the extreme singular values of (structured) Gaussian random matrices. The upper bound is constructive. It is proven for the worst case error of a least squares estimator.