Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Optimal Linear B-splines Approximation via Kolmogorov Superposition Theorem and its Application

Ming-Jun Lai, Zhaiming Shen

TL;DR

This work leverages the Kolmogorov superposition theorem to build a high-dimensional function-approximation framework based on linear KB-splines, achieving an optimal $O\left(\frac{1}{n^2}\right)$ rate with a dimensionally linear constant for a dense Kolmogorov-Hölder subclass. By denoising KB-splines into smooth LKB-splines via tensor-product spline methods, it extends accurate approximation to $d=4$ and $d=6$ while keeping the parameter count at $O(nd)$, effectively mitigating the curse of dimensionality for the targeted function class. The approach is validated through extensive numerical experiments and demonstrated to be applicable to solving the Poisson equation numerically, highlighting practical impact for high-dimensional approximation and PDE contexts. Future work aims to broaden the framework with Kolmogorov-Fourier bases and to address discontinuities, expanding the range of applicable problems.

Abstract

We propose a new approach for approximating functions in $C([0,1]^d)$ via Kolmogorov superposition theorem (KST) based on the linear spline interpolation of the outer function in the Kolmogorov representation. We improve the results in \cite{LaiShenKST21} by showing that the optimal rate of approximation based on our proposed approach is $O(\frac{1}{n^2})$, where $n$ denotes the number of knots over $[0,1]$. Furthermore, the approximation constant scales linearly with the dimension $d$. We show that there exists a dense subclass in $C([0,1]^d)$ whose approximation can achieve such optimal rate, and the number of parameters needed in such approximation is at most $O(nd)$. Thus, there is no curse of dimensionality when approximating functions in this subclass. Moreover, for $d\geq 4$, we apply tensor product spline denoising technique to denoise KB-splines and get the smooth LKB-splines. We use LKB-splines as basis to approximate functions for the cases when $d=4$ and $d=6$, which extends the results in \cite{LaiShenKST21}. In addition, we validate via numerical experiments that fewer than $O(nd)$ function values are needed to achieve the rate $O(\frac{1}{n^β})$ for some $β>0$ based on the smoothness of the outer function. Finally, we demonstrate that our approach can be applied to numerically solving partial differential equation such as the Poisson equation with accurate results.

The Optimal Linear B-splines Approximation via Kolmogorov Superposition Theorem and its Application

TL;DR

This work leverages the Kolmogorov superposition theorem to build a high-dimensional function-approximation framework based on linear KB-splines, achieving an optimal rate with a dimensionally linear constant for a dense Kolmogorov-Hölder subclass. By denoising KB-splines into smooth LKB-splines via tensor-product spline methods, it extends accurate approximation to and while keeping the parameter count at , effectively mitigating the curse of dimensionality for the targeted function class. The approach is validated through extensive numerical experiments and demonstrated to be applicable to solving the Poisson equation numerically, highlighting practical impact for high-dimensional approximation and PDE contexts. Future work aims to broaden the framework with Kolmogorov-Fourier bases and to address discontinuities, expanding the range of applicable problems.

Abstract

We propose a new approach for approximating functions in via Kolmogorov superposition theorem (KST) based on the linear spline interpolation of the outer function in the Kolmogorov representation. We improve the results in \cite{LaiShenKST21} by showing that the optimal rate of approximation based on our proposed approach is , where denotes the number of knots over . Furthermore, the approximation constant scales linearly with the dimension . We show that there exists a dense subclass in whose approximation can achieve such optimal rate, and the number of parameters needed in such approximation is at most . Thus, there is no curse of dimensionality when approximating functions in this subclass. Moreover, for , we apply tensor product spline denoising technique to denoise KB-splines and get the smooth LKB-splines. We use LKB-splines as basis to approximate functions for the cases when and , which extends the results in \cite{LaiShenKST21}. In addition, we validate via numerical experiments that fewer than function values are needed to achieve the rate for some based on the smoothness of the outer function. Finally, we demonstrate that our approach can be applied to numerically solving partial differential equation such as the Poisson equation with accurate results.
Paper Structure (12 sections, 9 theorems, 39 equations, 4 figures, 4 tables)

This paper contains 12 sections, 9 theorems, 39 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Kolmogorov-Hölder class
  3. Kolmogorov-polynomials
  4. Linear KB-splines and LKB-splines
  5. Tensor product approximation and denoising
  6. Tensor product approximation of Bernstein polynomial
  7. Spline denoising
  8. Tensor product of spline denoising
  9. Numerical experiments for LKB-splines based approximation
  10. Application to numerically solving the Poisson equation
  11. Numerical results
  12. Conclusion and future research

Key Result

Theorem 1

Let $f\in C([0,1]^d)$, then there exist continuous functions $g_q:\mathbb{R}\to\mathbb{R}$ and $\phi_{qp}:[0,1]\to\mathbb{R}$ such that

Figures (4)

  • Figure 1: Examples of Kolmogorov-monomials (Top Row: $p_n(x)=x, x^2$. Bottom Row: $p_n(x)=x^4, x^8$).
  • Figure 2: Some examples of linear LKB-splines (the first and third columns) which are the smoothed version of the corresponding linear KB-splines (the second and fourth columns).
  • Figure 3: Plots of convergence rate on the Log-log scale in 4D and 6D based on pivotal dataset.
  • Figure 4: Number of pivotal locations (vertical axis) against $n$ (horizontal axis) in 4D (left) and in 6D (right).

Theorems & Definitions (18)

  • Theorem 1: Kolmogorov Superposition Theorem -- original version Kolmogorov57
  • Theorem 2: Kolmogorov Superposition Theorem -- Lorentz's version Lorentz66
  • Remark 1
  • Theorem 3
  • proof
  • Theorem 4: Kolmogorov-Weierstrass Theorem
  • proof
  • Remark 2
  • Lemma 1
  • Remark 3
  • ...and 8 more