Boosted optimal weighted least-squares

TL;DR

This work tackles efficient, stable function approximation in $V_m$ from few evaluations by boosting weighted least-squares with an optimal sampling measure, resampling, and greedy subsampling. The method (BLS) ensures almost-sure stability with $n$ near the interpolation regime $n\approx m$ and offers quasi-optimal error bounds in expectation, even under noise. Numerical experiments show that BLS matches or outperforms state-of-the-art interpolation and standard weighted LS in accuracy with significantly fewer samples, while maintaining robustness to measurement noise. The approach provides a practical surrogate-modeling tool for uncertainty quantification and design optimization where function evaluations are costly.

Abstract

This paper is concerned with the approximation of a function $u$ in a given approximation space $V_m$ of dimension $m$ from evaluations of the function at $n$ suitably chosen points. The aim is to construct an approximation of $u$ in $V_m$ which yields an error close to the best approximation error in $V_m$ and using as few evaluations as possible. Classical least-squares regression, which defines a projection in $V_m$ from $n$ random points, usually requires a large $n$ to guarantee a stable approximation and an error close to the best approximation error. This is a major drawback for applications where $u$ is expensive to evaluate. One remedy is to use a weighted least squares projection using $n$ samples drawn from a properly selected distribution. In this paper, we introduce a boosted weighted least-squares method which allows to ensure almost surely the stability of the weighted least squares projection with a sample size close to the interpolation regime $n=m$. It consists in sampling according to a measure associated with the optimization of a stability criterion over a collection of independent $n$-samples, and resampling according to this measure until a stability condition is satisfied. A greedy method is then proposed to remove points from the obtained sample. Quasi-optimality properties are obtained for the weighted least-squares projection, with or without the greedy procedure. The proposed method is validated on numerical examples and compared to state-of-the-art interpolation and weighted least squares methods.

Figures (7)

• Figure 1: Improvement of the bound for different values of $m$ and $M$.
• Figure 2: Probability density function of $Z_{\bm{x}^n} = \Vert \bm{G}_{\bm{x}^n} -\bm{I} \Vert_2$ for $V_m =\mathbb{P}_5$, with $\delta = 0.9$ and $n = 100$.
• Figure 3: Probability density function of $Z_{\bm{x}^n} = \Vert \bm{G}_{\bm{x}^n} -\bm{I} \Vert_2$ for $V_m =\mathbb{P}_5$, with $\delta = 0.9$ and $n=100$.
• Figure 4: Distributions of the $x^{(i)}, i=1, \hdots, 10$, with $\bm{x}^{10}$ sampled from the c-BLS method for $V_m = \mathbb{P}_5$ and $\mu$ the Gaussian measure.
• Figure 5: Distributions of the $x^{(i)}, i=1, \hdots, 10$, with $\bm{x}^{10}$ sampled from the c-BLS method for $V_m = \mathbb{P}_5$ and $\mu$ the uniform measure.

Theorems & Definitions (35)

• Lemma 2.1
• proof
• Theorem 2.2
• proof
• Theorem 2.3: Cohen2016
• proof
• Theorem 2.4
• proof
• Theorem 2.5
• proof