Finite Element Representation Network (FERN) for Operator Learning with a Localized Trainable Basis

TL;DR

The paper tackles operator learning for PDEs by addressing the inefficiency of global learnable bases in neural operators. It introduces the Finite Element Representation Network (FERN), which constructs adaptive, locally supported hat-function bases inside a shallow network using two-parameter ReLU representations, enabling exact basis assembly and adaptive refinement during training. The approach yields competitive or superior accuracy to strong baselines like DeepONet and POD across seven PDE families, while dramatically reducing the number of trainable parameters and demonstrating clear basis adaptivity to localized features such as bumps and shocks. This work bridges finite element concepts with neural operator learning, offering an interpretable and efficient pathway for solving parametrized PDEs with localized phenomena.

Abstract

We propose a finite-element local basis-based operator learning framework for solving partial differential equations (PDEs). Operator learning aims to approximate mappings from input functions to output functions, where the latter are typically represented using basis functions. While non-learnable bases reduce training costs, learnable bases offer greater flexibility but often require deep network architectures with a large number of trainable parameters. Existing approaches typically rely on deep global bases; however, many PDE solutions exhibit local behaviors such as shocks, sharp gradients, etc., and in parametrized PDE settings, these localized features may appear in different regions of the domain across different training and testing samples. Motivated by the use of local bases in finite element methods (FEM) for function approximation, we develop a shallow neural network architecture that constructs adaptive FEM bases. By adopting suitable activation functions, such as ReLU, the FEM bases can be assembled exactly within the network, introducing no additional approximation error in the basis construction process. This design enables the learning procedure to naturally mimic the adaptive refinement mechanism of FEM, allowing the network to discover basis functions tailored to intrinsic solution features such as shocks. The proposed learnable adaptive bases are then employed to represent the solution (output function) of the PDE. This framework reduces the number of trainable parameters while maintaining high approximation accuracy, effectively combining the adaptivity of FEM with the expressive power of operator learning. To evaluate performance, we validate the proposed method on seven families of PDEs with diverse characteristics, demonstrating its accuracy, efficiency, and robustness.

Figures (22)

• Figure 1: A hat function demonstration.
• Figure 2: Demonstration of the predictions for the Allen Cahn example (one degree of freedom). The relative prediction error for left and right examples are $3.70\%$ and $4.00\%$ respectively. The average and variance of the relative error for 190 testing samples are $3.61\%\pm 1.3\%$.
• Figure 3: Allen-Cahn equation with one degree of freedom. Left: Histogram of all learned FEM basis centers $a_k$, which are initially uniformly distributed over $[0, 1]$. Right: Histogram of all learned support sizes $h_k$, all of which are initialized to $0.05$. Since the distinctive features of the solution landscapes, such as shocks, bumps, rapid decay, and rapid growth, are evenly distributed across the domain for different samples (see Figure \ref{['fig_ac_sol_pred']}), most learned hat functions shrink to smaller supports after training (right panel), exhibiting localized behavior. Meanwhile, their centers remain evenly distributed to effectively capture these features across various samples (left panel). This behavior is consistently observed for other numbers of basis functions tested; see Figure \ref{['fig_ac_2dofs_h_moving']} for an illustration.
• Figure 4: Predicted solutions for the Allen–Cahn equation with two degrees of freedom. The relative prediction errors for the four cases (displayed from left to right, top to bottom) are $2.21\%$, $2.23\%$, $3.90\%$, and $4.76\%$, respectively.
• Figure 5: Allen-Cahn equation with two degrees of freedom. Left: Histogram of all learned FEM basis centers $a$, which are initially uniformly distributed over $[0, 1]$. Right: Histogram of all learned support sizes $h$, all of which are initialized to $0.05$. Since the distinctive features of the solution landscapes, such as shocks, bumps, rapid decay, and rapid growth, are evenly distributed across the domain for different samples (see Figure \ref{['fig_ac_2dofs_sol_pred']}), most learned functions shrink to smaller supports after training (right picture), exhibiting localized behavior. However, their centers remain evenly distributed to effectively capture these features across various samples (left picture). This behavior is consistently observed across different numbers of basis functions $N$ tested (see Figure \ref{['fig_ac_2dofs_h_moving']}).