Non-Intrusive Hyperreduction by a Physics-Augmented Neural Network with Second-Order Sobolev Training

TL;DR

The results indicate quick divergence of physics-augmented neural networks for extrapolation, preventing its deployment, and a best model is compared to reference work and the trajectory piecewise linear approach.

Abstract

The finite element method is an indispensable tool in engineering, but its computational complexity prevents applications for control or at system-level. Model order reduction bridges this gap, creating highly efficient yet accurate surrogate models. Reducing nonlinear setups additionally requires hyperreduction. Compatibility with commercial finite element software requires non-intrusive methods based on data. Methods include the trajectory piecewise linear approach, or regression, typically via neural networks. Important aspects for these methods are accuracy, efficiency, generalization, including desired physical and mathematical properties, and extrapolation. Especially the last two aspects are problematic for neural networks. Therefore, several studies investigated how to incorporate physical knowledge or desirable properties. A promising approach from constitutive modeling is physics augmented neural networks. This concept has been elegantly transferred to hyperreduction by Fleres et al. in 2025 and guarantees several desired properties, incorporates physics, can include parameters, and results in smaller architectures. We augment this reference work by second-order Sobolev training, i.e., using a function and its first two derivatives. These are conveniently accessible and promise improved performance. Further modifications are proposed and studied. While Sobolev training does not meet expectations, several minor changes improve accuracy by up to an order of magnitude. Eventually, our best model is compared to reference work and the trajectory piecewise linear approach. The comparison relies on the same numerical case study as the reference work and additionally emphasizes extrapolation due to its critical role in typical applications. Our results indicate quick divergence of physics-augmented neural networks for extrapolation, preventing its deployment.

Figures (16)

• Figure 1: ICNN architecture. Quantities marked with a bar are constrained to be non-negative. The $f(\bm{x}_r)$-term represents a quadratic energy layer, i.e., energy as it would be computed for a linear system. Note that bias terms are not shown and are not subject to constraints. The final output $z_k$ correspond to an energy-like quantity $\hat{e}$.
• Figure 2: FEM model of the numerical case study. A cantilever beam is fixed at one end and subject to a tip load at the free end.
• Figure 3: Side view of beam deformation for all load steps where color indicates the four load cases. Note that only the first load case colored in cyan is used for constructing or training the hyperreduced model while the other three constitute a challenging test.
• Figure 4: Reduced coordinates, strain energy, reduced force components, and reduced tangent stiffness components vs. load magnitude. Fewer matrix components are included due to symmetry. Note the stiffening behavior and the different orders of magnitude of energy, force, and tangent stiffness.
• Figure 5: Statistical evaluation of model performances with respect to the validation error: each box summarizes the $10$ initializations per variant. Each of the six subplots corresponds to a combination of architecture and input scaling, while the three learning rates constitute the x-axis per subplot.