Loss Landscape Geometry and the Learning of Symmetries: Or, What Influence Functions Reveal About Robust Generalization

TL;DR

The paper tackles whether neural PDE emulators genuinely learn physical symmetries or merely memorize data by introducing an influence-function–based diagnostic that measures orbit-wise gradient coherence across symmetry actions. It applies this framework to autoregressive emulators (UNet and ViT) trained on 2D compressible Euler and Navier–Stokes data, showing that forward-equivalence can coexist with incoherent learning signals across symmetry orbits, depending on architecture and data. The study reveals a trade-off: models with strong symmetry biases exhibit uniform gradient coupling and principled generalization but slower optimization, whereas flexible models converge quickly with strong but localized orbit-wise responses, risking symmetry-incompatible basins. The diagnostic provides a principled tool for auditing symmetry learning in scientific ML, guiding data augmentation and architectural choices to foster robust, physically consistent generalization.

Abstract

We study how neural emulators of partial differential equation solution operators internalize physical symmetries by introducing an influence-based diagnostic that measures the propagation of parameter updates between symmetry-related states, defined as the metric-weighted overlap of loss gradients evaluated along group orbits. This quantity probes the local geometry of the learned loss landscape and goes beyond forward-pass equivariance tests by directly assessing whether learning dynamics couple physically equivalent configurations. Applying our diagnostic to autoregressive fluid flow emulators, we show that orbit-wise gradient coherence provides the mechanism for learning to generalize over symmetry transformations and indicates when training selects a symmetry compatible basin. The result is a novel technique for evaluating if surrogate models have internalized symmetry properties of the known solution operator.

Figures (20)

• Figure 1: Dihedral group equivariance error on Navier-Stokes (NS) data. Relative SMSE evaluated on NS trajectories. Points denote medians over seeds and test examples; ranges indicate variability. Deviations from unity quantify dihedral symmetry breaking. For the analogous result involving CE data, see \ref{['fig:err_D4_CE']}.
• Figure 2: Dihedral group influence on NS data. Influence between an input and its square group rotated state on NS data. Points denote medians; ranges summarize inter-seed and inter-example variability, measuring coupling of learning dynamics along dihedral orbits. For the analogous result involving CE data, see \ref{['fig:inf_D4_CE']}.
• Figure 3: Horizontal-translation equivariance error on CE data. Third-quantile (Q3) relative SMSE as a function of horizontal translation. Markers denote medians across seeds; ranges summarize variability over seeds and test examples, emphasizing upper-tail symmetry breaking. For analogous results with NS data, see \ref{['fig:err_ZH_NS']}.
• Figure 4: Vertical-translation equivariance error on CE data. Third-quantile relative SMSE under vertical translations. Markers denote medians across seeds; ranges summarize inter-seed and inter-example variability, isolating upper-tail error under symmetry perturbations. For analogous results with NS data, see \ref{['fig:err_ZV_NS']}.
• Figure 5: Horizontal-translation influence on CE data. Influence between an input and its horizontally translated state; structured dependence on translation distance reveals spatial coupling of learning dynamics. Markers denote medians; ranges indicate variability over seeds and test examples. For analogous results with NS data, see \ref{['fig:inf_ZH_NS']}.