Sparse RBF Networks for PDEs and nonlocal equations: function space theory, operator calculus, and training algorithms

TL;DR

Theoretical and computational results consolidate and generalize the theoretical and computational framework of SparseRBFnet, supporting accurate sparse representations with efficient operator evaluation and offering theory-grounded guidance for algorithmic and modeling choices.

Abstract

This work presents a systematic analysis and extension of the sparse radial basis function network (SparseRBFnet) previously introduced for solving nonlinear partial differential equations (PDEs). Based on its adaptive-width shallow kernel network formulation, we further investigate its function-space characterization, operator evaluation, and computational algorithm. We provide a unified description of the solution space for a broad class of radial basis functions (RBFs). Under mild assumptions, this space admits a characterization as a Besov space, independent of the specific kernel choice. We further demonstrate how the explicit kernel-based structure enables quasi-analytical evaluation of both differential and nonlocal operators, including fractional Laplacians. On the computational end, we study the adaptive-width network and related three-phase training strategy through a comparison with variants concerning the modeling and algorithmic details. In particular, we assess the roles of second-order optimization, inner-weight training, network adaptivity, and anisotropic kernel parameterizations. Numerical experiments on high-order, fractional, and anisotropic PDE benchmarks illustrate the empirical insensitivity to kernel choice, as well as the resulting trade-offs between accuracy, sparsity, and computational cost. Collectively, these results consolidate and generalize the theoretical and computational framework of SparseRBFnet, supporting accurate sparse representations with efficient operator evaluation and offering theory-grounded guidance for algorithmic and modeling choices.

Figures (7)

• Figure 1: Comparison between models trained to solve an elliptic PDE using different algorithmic variants. In each panel, the left plot shows the trained model on the domain, while the right plot shows the residual on the domain, together with the trained weights. Each $\star$ represents the location parameter $y_{n}$ of a learned kernel node, and the dotted circle around has radius $\sigma_{n}$.
• Figure 2: Performance comparison of fixed-size models with varying budget $N_{\mathrm{fixed}}$ (with $\ell^{1}$ and $\ell^{2}$ regularization, respectively) against the adaptive method (ours). Results aggregated from 10 runs. The regularization parameter is set to $\alpha = 10^{-3}$ for the $\ell^{1}$-regularized case and to $\alpha = 10^{-5}$ for the $\ell^{2}$-regularized case. (A) Relative test error versus fixed budget $N_{\mathrm{fixed}}$. The dashed horizontal line indicates the adaptive network benchmark (ours). Runs with relative $L^{2}$ error $(\mathrm{err}^{\mathrm{rel}}_{2})$ below $30.0\%$ are classified as successful and are retained for statistical aggregation. The corresponding success rate is indicated near each data point; (B) Runtime versus fixed budget $N_{\mathrm{fixed}}$. The dashed horizontal line indicates the adaptive network benchmark.
• Figure 3: Convergence of loss of full solver and outer only solver ($\alpha=1e\text{-}4$ with Gaussian kernel).
• Figure 4: Exact solutions and error contours for the 2D fractional Poisson equation on the unit disk with different fractional orders ($\beta$). Each $\star$ in the error contour represents the location parameter $y_{n}$ of a learned kernel node, and the dotted circle around has radius $\sigma_{n}$.
• Figure 5: Comparison of isotropic and anisotropic kernels on the anisotropic viscous Eikonal equation. Each $\star$ in the error contour represents the location parameter $y_{n}$ of a learned kernel node. In the isotropic case, the dotted circle around each $\star$ has radius $\sigma_{n}$; in the anisotropic case, the dotted ellipsoid represents the level set $|\Sigma_{n}^{-1}(\cdot - y_{n})| = 1$.

Theorems & Definitions (9)

• Remark 2
• Theorem 3
• Remark 4
• Lemma 5
• proof
• Lemma 6
• proof
• Lemma 7
• proof