Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the power of iid information for linear approximation

Mathias Sonnleitner, Mario Ullrich

TL;DR

The paper addresses the power of iid information for deterministic worst-case linear approximation by framing a general weighted least-squares approach on a chosen subspace and proving high-probability error bounds that connect to random matrices and concentration phenomena. It provides a versatile, architecture-agnostic framework that extends from $L_2$-approximation to general norms via embedding assumptions, yielding concrete results for Hilbert spaces, Sobolev spaces, and Gaussian information, including sharp oversampling results and subsampling tricks. The work highlights both the near-optimality of iid information in many settings (often with an $O(n\log n)$ sampling scale) and the limits (e.g., Fourier-type information requiring oversampling, dimension-related curse) and shows how these insights translate into practical sampling strategies and learning perspectives. Collectively, the results advance understanding of how random information acts as a universal, robust tool in information-based complexity, with implications for high-dimensional approximation, RKHS methods, and learning theory.

Abstract

This survey is concerned with the power of random information for approximation in the (deterministic) worst-case setting, with special emphasis on information consisting of functionals selected independently and identically distributed (iid) at random on a class of admissible information functionals. We present a general result based on a weighted least squares method and derive consequences for special cases. Improvements are available if the information is ``Gaussian'' or if we consider iid function values for Sobolev spaces. We include open questions to guide future research on the power of random information in the context of information-based complexity.

On the power of iid information for linear approximation

TL;DR

The paper addresses the power of iid information for deterministic worst-case linear approximation by framing a general weighted least-squares approach on a chosen subspace and proving high-probability error bounds that connect to random matrices and concentration phenomena. It provides a versatile, architecture-agnostic framework that extends from -approximation to general norms via embedding assumptions, yielding concrete results for Hilbert spaces, Sobolev spaces, and Gaussian information, including sharp oversampling results and subsampling tricks. The work highlights both the near-optimality of iid information in many settings (often with an sampling scale) and the limits (e.g., Fourier-type information requiring oversampling, dimension-related curse) and shows how these insights translate into practical sampling strategies and learning perspectives. Collectively, the results advance understanding of how random information acts as a universal, robust tool in information-based complexity, with implications for high-dimensional approximation, RKHS methods, and learning theory.

Abstract

This survey is concerned with the power of random information for approximation in the (deterministic) worst-case setting, with special emphasis on information consisting of functionals selected independently and identically distributed (iid) at random on a class of admissible information functionals. We present a general result based on a weighted least squares method and derive consequences for special cases. Improvements are available if the information is ``Gaussian'' or if we consider iid function values for Sobolev spaces. We include open questions to guide future research on the power of random information in the context of information-based complexity.
Paper Structure (24 sections, 22 theorems, 139 equations)

This paper contains 24 sections, 22 theorems, 139 equations.

Table of Contents

  1. Introduction
  2. What this survey is not about
  3. Notation:
  4. Some benchmarks of optimal approximation
  5. Approximation numbers
  6. Kolmogorov widths
  7. Gelfand numbers
  8. Sampling numbers
  9. Approximation based on iid information
  10. $L_2$-approximation and least squares methods
  11. Hilbert spaces and (random) matrices
  12. General classes of functions
  13. Approximation in general norms
  14. Sharpness for $L_2$-approximation
  15. Applications and improvements
  16. ...and 9 more sections

Key Result

Proposition 2

Let $H\subset L_2$ be a normed space of functions on $D$, and, for $n\le N$, let $V_n$ be a $n$-dimensional subspace of $H$, and $\ell_1,\dots,\ell_N$ be linear functionals on $H$. Assume that for some $\alpha>0$ and some weights $w_1,\dots,w_N>0$. Then, for all $f\in H$ and $g\in V_n$, the algorithm from eq:alg with the corresponding $V_n$, $\ell_i$ and $w_i$ is well-defined and satisfies

Theorems & Definitions (36)

  • Remark 1
  • Proposition 2
  • Corollary 3
  • proof : Proof of Proposition \ref{['prop:LS']}
  • Proposition 4
  • Theorem 5
  • proof : Proof of Proposition \ref{['prop:LS-H']}
  • Lemma 6
  • proof : Proof of Theorem \ref{['thm:general-H']}
  • Remark 7
  • ...and 26 more