Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reconstruction of frequency-localized functions from pointwise samples via least squares and deep learning

A. Martina Neuman, Andres Felipe Lerma Pineda, Jason J. Bramburger, Simone Brugiapaglia

TL;DR

<3-5 sentence high-level summary> This work studies reconstructing frequency-localized functions from pointwise samples using two complementary approaches: least-squares regression with a Slepian (prolate spheroidal) basis and deep neural networks that approximate Slepian representations. It provides rigorous sample-complexity results for LS in low dimensions and a practical existence theorem showing that neural networks, trained with a least-squares-like objective, can achieve comparable approximation guarantees. The paper also delivers numerical comparisons in 1D and 2D, demonstrates a Slepian-based neural initialization to boost performance, and discusses theoretical and practical gaps between theory and implementation. Overall, it advances understanding of frequency-localized function recovery from scattered samples, leveraging spatiospectral concentration to guide both linear and nonlinear approximation strategies.

Abstract

Recovering frequency-localized functions from pointwise data is a fundamental task in signal processing. We examine this problem from an approximation-theoretic perspective, focusing on least squares and deep learning-based methods. First, we establish a novel recovery theorem for least squares approximations using the Slepian basis from uniform random samples in low dimensions, explicitly tracking the dependence of the bandwidth on the sampling complexity. Building on these results, we then present a recovery guarantee for approximating bandlimited functions via deep learning from pointwise data. This result, framed as a practical existence theorem, provides conditions on the network architecture, training procedure, and data acquisition sufficient for accurate approximation. To complement our theoretical findings, we perform numerical comparisons between least squares and deep learning for approximating one- and two-dimensional functions. We conclude with a discussion of the theoretical limitations and the practical gaps between theory and implementation.

Reconstruction of frequency-localized functions from pointwise samples via least squares and deep learning

TL;DR

<3-5 sentence high-level summary> This work studies reconstructing frequency-localized functions from pointwise samples using two complementary approaches: least-squares regression with a Slepian (prolate spheroidal) basis and deep neural networks that approximate Slepian representations. It provides rigorous sample-complexity results for LS in low dimensions and a practical existence theorem showing that neural networks, trained with a least-squares-like objective, can achieve comparable approximation guarantees. The paper also delivers numerical comparisons in 1D and 2D, demonstrates a Slepian-based neural initialization to boost performance, and discusses theoretical and practical gaps between theory and implementation. Overall, it advances understanding of frequency-localized function recovery from scattered samples, leveraging spatiospectral concentration to guide both linear and nonlinear approximation strategies.

Abstract

Recovering frequency-localized functions from pointwise data is a fundamental task in signal processing. We examine this problem from an approximation-theoretic perspective, focusing on least squares and deep learning-based methods. First, we establish a novel recovery theorem for least squares approximations using the Slepian basis from uniform random samples in low dimensions, explicitly tracking the dependence of the bandwidth on the sampling complexity. Building on these results, we then present a recovery guarantee for approximating bandlimited functions via deep learning from pointwise data. This result, framed as a practical existence theorem, provides conditions on the network architecture, training procedure, and data acquisition sufficient for accurate approximation. To complement our theoretical findings, we perform numerical comparisons between least squares and deep learning for approximating one- and two-dimensional functions. We conclude with a discussion of the theoretical limitations and the practical gaps between theory and implementation.

Paper Structure

This paper contains 31 sections, 19 theorems, 172 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Main contributions
  3. Paper outline
  4. Setup
  5. Bandlimited and Slepian functions
  6. The Slepian basis in one dimension.
  7. The Slepian basis in $d$ dimensions.
  8. Problem setting
  9. Main results
  10. Least squares approximation
  11. Practical existence theorem
  12. Discussion on further recovery guarantees for least squares
  13. Numerical experiments
  14. Training set, test set, and error metric.
  15. Slepian basis functions.
  16. ...and 16 more sections

Key Result

Theorem 3.1

Let $\beta,\delta\in (0,1)$, $d=1,2,3$, and $\Lambda=\Lambda^{\rm HC}_{n-1}$ be the $d$-dimensional hyperbolic cross of order $n$, where $n\in\mathbb{N}$ if $d=1,2$, and $n\geq 26$ if $d=3$. For $\mathsf{w}\geq 1$, let $\mathcal{S}_{\Lambda}$ be the linear span of $\{\varphi_{\mathsf{w},\vec{\nu}}\} then, with probability at least $1-\beta$, for every $f \in \mathcal{C}([-1,1]^d)$ and every noise

Figures (7)

  • Figure 1: Plot of Slepian functions (PSWFs) $\varphi_{16,j}$ with $\mathsf{w} = 16$ for odd (left) and even (right) indices $j = 0,1,2,3,4,5,6,7$.
  • Figure 2: Comparison of Least Squares approximations using different bases for the function $g_1(x) = e^{-\pi x^2}$ from $m$ random samples. The left panel shows results using $n=10$ basis functions and varying the number of samples, while the right panel shows results when the number of samples is fixed at $m=1000$ and the order parameter $n$ changes. Note that the results for Chebyshev and Legendre polynomials can be hard to distinguish because they are almost identical.
  • Figure 3: Comparison of least squares approximations using different bases for the function $g_2(x) = e^{ 4i \pi x}$ from $m$ random samples. The left panel shows results using $n=20$ basis functions and varying the number of samples, while the right panel shows results when the number of samples is fixed at $m=1000$ and the order parameter $n$ changes. Note that the results for Chebyshev and Legendre polynomials can be hard to distinguish because they are almost identical.
  • Figure 4: Comparison of Least Squares approximations using different bases for the function $g_3(x) = e^{-\pi(x^2+y^2+0.2xy)}$ from random samples. The left panel shows results using a hyperbolic cross $\Lambda^{\text{HC}}_{n-1}$ with $n=10$ and varying the number of samples, while the right panel shows results when the number of samples is fixed at $m=1000$ and the order parameter $n$ changes. Note that the results for Chebyshev and Legendre polynomials can be hard to distinguish because they are almost identical.
  • Figure 5: Training and test errors for neural networks with architecture $(1,1000,1000,10,1)$ approximating the function $f_1(x) = \cos(10x)e^{-\pi x^2}$ using different initialization strategies. The left panel shows the training error and the right panel shows the test error.
  • ...and 2 more figures

Theorems & Definitions (37)

  • Definition 2.1
  • Remark 2.2: Practical advantages of Slepian representations
  • Definition 2.3
  • Definition 2.4: PetV2018OptApproxReLU
  • Theorem 3.1: Recovery guarantee for least squares
  • Remark 3.2: Comparison with polynomial least squares approximation theory
  • Theorem 3.3: Practical existence theorem
  • Corollary 3.4: $L_{ {\bf u}}^{2}$-$L_{ {\bf u}}^{2}$ recovery guarantee in probability for least squares
  • Corollary 3.5: $L_{ {\bf u}}^{2}$-$L_{ {\bf u}}^{2}$ recovery guarantee in expectation for least squares
  • Proposition 5.1
  • ...and 27 more