Sparse FEONet: A Low-Cost, Memory-Efficient Operator Network via Finite-Element Local Sparsity for Parametric PDEs

TL;DR

FEONet learns the parameter-to-solution map on a finite-element space in an unsupervised way, but its cost grows with mesh refinement. The authors propose Sparse FEONet, a finite-element locality-guided sparsity pattern that dramatically reduces parameter count and memory while preserving accuracy, and provide a universal-approximation theorem and stability analysis. Across advection-diffusion-reaction, Helmholtz, Burgers, and irregular meshes, sparse FEONet converges on finer meshes where dense FEONet struggles or runs out of memory, with orders-of-magnitude fewer parameters. The work offers a principled integration of finite-element locality into operator learning, paving the way for scalable, data-efficient solvers for parametric PDEs (e.g., approximating $x \mapsto A^{-1}x$) and suggesting directions for extending sparsity to other architectures.

Abstract

In this paper, we study the finite element operator network (FEONet), an operator-learning method for parametric problems, originally introduced in J. Y. Lee, S. Ko, and Y. Hong, Finite Element Operator Network for Solving Elliptic-Type Parametric PDEs, SIAM J. Sci. Comput., 47(2), C501-C528, 2025. FEONet realizes the parameter-to-solution map on a finite element space and admits a training procedure that does not require training data, while exhibiting high accuracy and robustness across a broad class of problems. However, its computational cost increases and accuracy may deteriorate as the number of elements grows, posing notable challenges for large-scale problems. In this paper, we propose a new sparse network architecture motivated by the structure of the finite elements to address this issue. Throughout extensive numerical experiments, we show that the proposed sparse network achieves substantial improvements in computational cost and efficiency while maintaining comparable accuracy. We also establish theoretical results demonstrating that the sparse architecture can approximate the target operator effectively and provide a stability analysis ensuring reliable training and prediction.

Figures (10)

• Figure 1: Schematic overview of the Finite Element Operator Network (FEONet) structure.
• Figure 2: Comparison of training losses between FC (dense) and sparsely-connected (sparse) FEONet for the 2D advection-diffusion-reaction equation across different resolutions $N_h=225, 961, \text{and} \ 3{,}969$. As $N_h$ increases, the dense connected network fails to converge while the sparse network shows stable convergence.
• Figure 3: Comparison of weight connectivity in fully connected and sparse layers ($C_\ell = 1$).
• Figure 4: Propagation from node 2 in the sparse network with $C_\ell = 1$ and $N_h = 25$. Blue shows the set reached after successive layers; gray shows other admissible connections.
• Figure 5: Sparse weight matrix patterns: (a) random ($N_h=225$), (b) our method ($N_h = 225$, $C_\ell = 4$), (c) random ($N_h = 961$), and (d) our method ($N_h = 961$, $C_\ell = 6$). The plot of random connectivity patterns is selected from one of the 10 random seeds.

Theorems & Definitions (23)

• Definition 4.1: Simple undirected graph
• Definition 4.2: Adjacency
• Definition 4.3: Path and graph distance
• Definition 4.4: Connected vertices and connected graph
• Definition 4.5: Connected component
• Definition 4.6
• Definition 4.7
• Theorem 4.8
• Proposition 4.9
• proof