BumpNet: A Sparse Neural Network Framework for Learning PDE Solutions

TL;DR

BumpNet introduces a sparse, interpretable basis-expansion approach for PDE learning by constructing adjustable localized bumps from sigmoid activations. The framework yields three practical instantiations—Bump-PINN for PDE solving, Bump-EDNN for time-evolution, and Bump-DeepONet for operator learning—each achieving strong accuracy with substantially fewer parameters than traditional neural networks. A simple amplitude-based pruning strategy enables h-adaptivity, concentrating resources where gradients are large and accelerating training. Across multiple benchmarks, BumpNet demonstrates superior parameter efficiency and competitive or superior accuracy, with mesh-free implementation and scalable performance for forward, inverse, and operator-learning tasks.

Abstract

We introduce BumpNet, a sparse neural network framework for PDE numerical solution and operator learning. BumpNet is based on meshless basis function expansion, in a similar fashion to radial-basis function (RBF) networks. Unlike RBF networks, the basis functions in BumpNet are constructed from ordinary sigmoid activation functions. This enables the efficient use of modern training techniques optimized for such networks. All parameters of the basis functions, including shape, location, and amplitude, are fully trainable. Model parsimony and h-adaptivity are effectively achieved through dynamically pruning basis functions during training. BumpNet is a general framework that can be combined with existing neural architectures for learning PDE solutions: here, we propose Bump-PINNs (BumpNet with physics-informed neural networks) for solving general PDEs; Bump-EDNN (BumpNet with evolutionary deep neural networks) to solve time-evolution PDEs; and Bump-DeepONet (BumpNet with deep operator networks) for PDE operator learning. Bump-PINNs are trained using the same collocation-based approach used by PINNs, Bump-EDNN uses a BumpNet only in the spatial domain and uses EDNNs to advance the solution in time, while Bump-DeepONets employ a BumpNet regression network as the trunk network of a DeepONet. Extensive numerical experiments demonstrate the efficiency and accuracy of the proposed architecture.

Figures (10)

• Figure 1: Network architecture of a 2D BumpNet basis module, assuming the tanh sigmoid. The output over the domain is a 2D bump, as seen in Fig. \ref{['fig-bump-vis']}.
• Figure 2: Line profile of a 2D bump in (a) and its derivative (b) for varying sharpness factor value. The bump is located the origin with unit sides. The bumps plotted in (c) and (d) have a sharpness factor of $0.1$.
• Figure 3: Network architecture of a Bump-DeepONet.
• Figure 4: Helmholtz equation with BumpNet-PINN.
• Figure 5: BumpNet-PINN for the Poisson equation.