Christoffel Adaptive Sampling for Sparse Random Feature Expansions

Abstract

Random Feature Models (RFMs) have become a powerful tool for approximating multivariate functions and solving partial differential equations efficiently. Sparse Random Feature Expansions (SRFE) improve traditional RFMs by incorporating sparsity, making it particularly effective in data-scarce settings. In this work, we integrate active learning with sparse random feature approximations to improve sampling efficiency. Specifically, we incorporate the Christoffel function to guide an adaptive sampling process, dynamically selecting informative sample points based on their contribution to the function space. This approach optimizes the distribution of sample points by leveraging the Christoffel function associated with an iteratively-chosen basis obtained by the sparse recovery solver. We conduct numerical experiments comparing adaptive and nonadaptive sampling strategies with the SRFE framework and examine their accuracy for various function approximation tasks. Overall, our results demonstrate the advantages of adaptive sampling in maintaining high accuracy while reducing sample complexity for SRFE, highlighting its potential for scientific computing tasks where data is expensive to acquire.

Figures (7)

• Figure 1: Diagnostics of the MH sampling algorithm with the CS measure \ref{['measurechris']} with $\rho = \mathcal{N}(0,\sigma^2 I)$. The parameters are $s=100$ (number of random features used for computing the Christoffel function), $d = 1$, $\sigma=1$, $\sigma_1=5$, $B=2000$, total samples total$=5000$, thinning $T=5$.
• Figure 2: Diagnostics of the MH sampling algorithm with the CS measure \ref{['measurechris']} with $\rho = \mathcal{N}(0,\sigma^2 I)$. The parameters are s = 100, $d$ = 2, $\sigma$ = 1, $\sigma_1$ = 5, burn-in $B = 5000$, total samples total = 5000, thinning $T= 15$.
• Figure 3: Diagnostics of the MH sampling algorithm with the CS measure \ref{['measurechris']} with $\rho = \mathrm{Exp}(\lambda)$, $\lambda=1e-3$. The parameters are $s = 100$, $d$ = 1, $\sigma$ = 1, $\sigma_1$ = 1.5, $\sigma_w=10^{-3}$, burn-in $B = 5000$, total samples $\texttt{total}= 5000$, thinning $T= 30$.
• Figure 4: Diagnostics of the MH sampling algorithm with the CS measure \ref{['measurechris']} with $\rho = \mathrm{Exp}(\lambda)$, $\lambda=1e-3$. The parameters are s = 100, $d$ = 2, $\sigma = 1$, $\sigma_1 = 1.5$, $\sigma_w=10^{-3}$, initial mean $x_0 = 1$, burn-in $B = 5000$, total samples $\texttt{total}= 5000$, thinning $T = 5$.
• Figure 5: CAS-SRFE and NAS-SRFE approximation errors versus $m$ for $\rho = \mathcal{N}(0,\sigma^2 I)$ with $\sigma = 1$. All experiments use sample sizes m = [100: 100: 1000]. Dimensions and number of random features are noted in each subfigure.