Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence and Near-optimal Sampling for Multivariate Function Approximations in Irregular Domains via Vandermonde with Arnoldi

Wenqi Zhu, Yuji Nakatsukasa

TL;DR

This work extends the univariate Vandermonde with Arnoldi (V+A) method to multivariate, irregular domains, formulating a least-squares framework that builds discrete orthogonal bases on the domain via Arnoldi orthogonalization. It proves that, for a broad class of domains, the V+A LS achieves near-optimal accuracy with M = O(N^2) equispaced points or M = O(N^2 log N) random points, and establishes spectral convergence rates tied to the smoothness of the target function. A substantial advancement is the introduction of VA+Weight, a weighted LS variant that requires only M = O(N log N) samples to attain near-optimal performance, with numerical evidence showing competitive stability and accuracy against state-of-the-art orthogonalization techniques. The results demonstrate robust multivariate polynomial approximation on irregular domains up to dimension d = 5, offering a practical pathway for sample-point design and stable high-degree approximations. Limitations related to the curse of dimensionality are acknowledged, with future work aimed at scalable algorithms, hyperbolic-cross spaces, and extensions to vector/matrix rational approximations.

Abstract

Vandermonde matrices are usually exponentially ill-conditioned and often result in unstable approximations. In this paper, we introduce and analyze the \textit{multivariate Vandermonde with Arnoldi (V+A) method}, which is based on least-squares approximation together with a Stieltjes orthogonalization process, for approximating continuous, multivariate functions on $d$-dimensional irregular domains. The V+A method addresses the ill-conditioning of the Vandermonde approximation by creating a set of discrete orthogonal bases with respect to a discrete measure. The V+A method is simple and general, relying only on the domain's sample points. This paper analyzes the sample complexity of {the least-squares approximation that uses the V+A method}. We show that, for a large class of domains, this approximation gives a well-conditioned and near-optimal $N$-dimensional least-squares approximation using $M=O(N^2)$ equispaced sample points or $M=O(N^2\log N)$ random sample points, independently of $d$. We provide a comprehensive analysis of the error estimates and the rate of convergence of the least-squares approximation that uses the V+A method. Based on the multivariate V+A techniques, we propose a new variant of the weighted V+A least-squares algorithm that uses only $M=O(N\log N)$ sample points to achieve a near-optimal approximation. {Our initial numerical results validate that the V+A least-squares approximation method provides well-conditioned and near-optimal approximations for multivariate functions on (irregular) domains. Additionally, the (weighted) least-squares approximation that uses the V+A method performs competitively with state-of-the-art orthogonalization techniques and can serve as a practical tool for selecting near-optimal distributions of sample points in irregular domains.

Convergence and Near-optimal Sampling for Multivariate Function Approximations in Irregular Domains via Vandermonde with Arnoldi

TL;DR

This work extends the univariate Vandermonde with Arnoldi (V+A) method to multivariate, irregular domains, formulating a least-squares framework that builds discrete orthogonal bases on the domain via Arnoldi orthogonalization. It proves that, for a broad class of domains, the V+A LS achieves near-optimal accuracy with M = O(N^2) equispaced points or M = O(N^2 log N) random points, and establishes spectral convergence rates tied to the smoothness of the target function. A substantial advancement is the introduction of VA+Weight, a weighted LS variant that requires only M = O(N log N) samples to attain near-optimal performance, with numerical evidence showing competitive stability and accuracy against state-of-the-art orthogonalization techniques. The results demonstrate robust multivariate polynomial approximation on irregular domains up to dimension d = 5, offering a practical pathway for sample-point design and stable high-degree approximations. Limitations related to the curse of dimensionality are acknowledged, with future work aimed at scalable algorithms, hyperbolic-cross spaces, and extensions to vector/matrix rational approximations.

Abstract

Vandermonde matrices are usually exponentially ill-conditioned and often result in unstable approximations. In this paper, we introduce and analyze the \textit{multivariate Vandermonde with Arnoldi (V+A) method}, which is based on least-squares approximation together with a Stieltjes orthogonalization process, for approximating continuous, multivariate functions on -dimensional irregular domains. The V+A method addresses the ill-conditioning of the Vandermonde approximation by creating a set of discrete orthogonal bases with respect to a discrete measure. The V+A method is simple and general, relying only on the domain's sample points. This paper analyzes the sample complexity of {the least-squares approximation that uses the V+A method}. We show that, for a large class of domains, this approximation gives a well-conditioned and near-optimal -dimensional least-squares approximation using equispaced sample points or random sample points, independently of . We provide a comprehensive analysis of the error estimates and the rate of convergence of the least-squares approximation that uses the V+A method. Based on the multivariate V+A techniques, we propose a new variant of the weighted V+A least-squares algorithm that uses only sample points to achieve a near-optimal approximation. {Our initial numerical results validate that the V+A least-squares approximation method provides well-conditioned and near-optimal approximations for multivariate functions on (irregular) domains. Additionally, the (weighted) least-squares approximation that uses the V+A method performs competitively with state-of-the-art orthogonalization techniques and can serve as a practical tool for selecting near-optimal distributions of sample points in irregular domains.
Paper Structure (20 sections, 9 theorems, 45 equations, 12 figures, 5 algorithms)

This paper contains 20 sections, 9 theorems, 45 equations, 12 figures, 5 algorithms.

Table of Contents

  1. Introduction and Overview of the Paper
  2. Related Work
  3. Univariate Vandermonde with Arnoldi
  4. Algorithm for Vandermonde with Arnoldi
  5. Applications and Numerical Examples for Univariate V+A
  6. Multivariate Vandermonde with Arnoldi
  7. Applications and Numerical Examples for Multivariate V+A
  8. Sample Complexity and Convergence of Multivariate V+A
  9. Markov Inequality
  10. Sampling
  11. Deterministic Sample Points
  12. Randomized Sample Points
  13. Convergence of Multivariate Vandermonde with Arnoldi
  14. Error Bound
  15. Convergence Rate
  16. ...and 5 more sections

Key Result

Lemma 4.1

(Markov Condition for $d=1$) For $x \in [-\zeta, \zeta]$ with $\zeta>0$, then $|p'({\bf x})| \le \frac{n^2}{\zeta} \max_{x \in [-\zeta, \zeta]}|p({\bf x})|.$

Figures (12)

  • Figure 1: The least squares approximation with the V+A method is computed using the univariant V+A (Algorithm \ref{['V+A Algo']}). The Vandermonde approximation is computed using polyfit/polyval provided in MATLAB.
  • Figure 2: Approximating $f(x_{(1)},x_{(2)})=\sin(\frac{x_{(1)}^2+x_{(2)}^2+x_{(1)}x_{(2)}}{5})$ on $[-1, 4] \times [-1, 6]$ using equispaced mesh with $M=N^2$.
  • Figure 3: Approximating $f(x_{(1)},x_{(2)})=\sin(\frac{x_{(1)}^2+x_{(2)}^2+x_{(1)}x_{(2)}}{5})$ on an elliptical domain (Domain $4$ in Figure \ref{['domain']}) using randomized mesh with $M=N^2\log N$ (See Theorem \ref{['Ramdon AM']} for more details on sample complexity). Here and throughout this paper, we plot the mean of $25$ random trials when the sample points are random. Unless otherwise stated, we use the total degree polynomial space for approximation in numerical experiments.
  • Figure 4: Left: Comparison of the orthogonal basis on the bounding domain approximation and the least squares approximation with the V+A method for $f(x_{(1)},x_{(2)})=\sin(\frac{x_{(1)}^2+x_{(2)}^2+x_{(1)}x_{(2)}}{5})$ on an elliptical domain. Right: Approximating non-smooth function $f(x_{(1)},x_{(2)})=|x_{(1)}-2||x_{(2)}-3|$ on an elliptical domain and $f(x)=|x-2|$ in $[0,4]$, both using ${\cal O}(N \log N)$ randomized sample points.
  • Figure 5: An illustration of the sample points where discrete orthogonal polynomials are generated for Example \ref{['example 3.4']}. In V+A, the bases are generated by sample points from the elliptical domain, represented by red dots in the first plot. In the orthogonal basis on the bounding domain approximation, the bases are Legendre polynomials on the tensor product domain generated by red dots in the second and third plots. The first and second plots have the same number of sample points. The first and third plots have the same number of sample points inside the elliptical domain.
  • ...and 7 more figures

Theorems & Definitions (38)

  • Remark 2.1
  • Example 2.1
  • Example 2.2
  • Example 2.3
  • Remark 3.1
  • Remark 3.2
  • Example 3.1
  • Example 3.2
  • Example 3.3
  • Example 3.4
  • ...and 28 more