High-dimensional sparse recovery from function samples Decoders, guarantees and instance optimality

TL;DR

This work develops a framework for high-dimensional sparse recovery of multivariate functions from function samples using nonlinear decoders (rLasso, OMP, CoSaMP). By leveraging bounded orthonormal systems and the RIP/NSP theory, it shows that reconstruction with a fixed sparsity and a chosen search space $V_J$ achieves $L_q$ accuracy close to the best $n$-term approximation, with sample complexity $m$ scaling as $m \gtrsim n\log^2 n\log|J|$, and proves that the log factor is necessary for instance-optimal guarantees. The analysis connects vector sparse recovery to function approximation in generalized Wiener spaces $\mathcal{A}_1$, and extends to general ONBs via a BOS transformation, providing rigorous bounds for random sampling widths. Numerical experiments on multivariate Fourier and Chebyshev systems validate the theory and illustrate when nonlinear decoders outperform linear approaches, highlighting practical applicability to high-dimensional function recovery and sampling width estimation.

Abstract

We investigate the reconstruction of multivariate functions from samples using sparse recovery techniques. For Square Root Lasso, Orthogonal Matching Pursuit, and Compressive Sampling Matching Pursuit, we demonstrate both theoretically and empirically that they allow us to recover functions from a small number of random samples. In contrast to Basis Pursuit Denoising, the deployed decoders only require a search space $V_J$ spanned by dictionary elements indexed by $J$ and a sparsity parameter $n$ to guarantee an $L_2$-approximation error decaying no worse than a best $n$-term approximation error and the truncation error with respect to the search space $V_J$ and the uniform norm. We show that this happens simultaneously for all admissible functions if the number of samples scales as $n\log^2 n\log |J|$, coming from known bounds for the RIP for matrices built upon bounded orthonormal systems. As a consequence, we obtain bounds for sampling widths in function classes. In addition, we establish lower bounds on the required sample complexity, which show that the log-factor in $\vert J \vert$ is indeed necessary to obtain such {\em instance-optimal} error guarantees. Finally, we conduct several numerical experiments to show that our theoretical bounds are reasonable and compare the discussed decoders in practice.

Figures (5)

• Figure 1: Example 1 with rLasso (top) and OMP (bottom): The frequencies form a hyperbolic cross and the samples are drawn uniformly from $[0,1]^5$. The errors are computed using the ground-truth Fourier coefficients. All values are up to sampling randomness.
• Figure 2: Example 2 with rLasso (top) and OMP (bottom): The frequencies form a hyperbolic cross and the samples are drawn uniformly from $[0,1]^7$. All values are up to sampling randomness.
• Figure 3: Function 3 with rLasso (top), OMP (middle) and CoSaMP (bottom): The search space is a hyperbolic cross and the samples are drawn according to the Chebyshev density on $[-1,1]^6$. Errors are computed using the ground-truth coefficients. All values are up to sampling randomness.
• Figure 3: Function 3: Approximation error depending on the number of samples for the rLasso, OMP and CoSaMP algorithm with fixed $J$. The plot is in log scale.
• Figure 4: Function 4 with rLasso (top) and OMP (bottom): The frequencies are chosen from the hyperbolic cross and the samples are drawn according to the Chebyshev density on $[-1,1]^6$.

Theorems & Definitions (46)

• Definition 2.1: rLasso
• Definition 2.2: OMP
• Definition 2.3: NSP and RIP
• Theorem 2.4: RIP implies $\ell_2$-robust NSP
• Proposition 2.5
• proof
• Proposition 2.6: FoRa13
• Proposition 2.7
• proof
• Theorem 2.8