Can Data-Driven Dynamics Reveal Hidden Physics? There Is A Need for Interpretable Neural Operators

TL;DR

The paper argues that neural operators can uncover hidden physics but lack robust interpretability, proposing a classification into spatial-domain and functional-domain models. It introduces effective receptive fields to analyze learned spatial dependencies and presents a multi-scale, dual-space perspective that combines global spectral learning with local spatial processing. Empirical insights from wave, Navier–Stokes, and other PDEs show spatial-domain models more faithfully capture spatial patterns, while dual-space architectures with physics priors offer improved performance and interpretability. The work advocates principled incorporation of known physics, including equivariant designs and physics-informed biases, as essential for generalization and reliable discovery of phenomena in data-driven dynamics.

Abstract

Recently, neural operators have emerged as powerful tools for learning mappings between function spaces, enabling data-driven simulations of complex dynamics. Despite their successes, a deeper understanding of their learning mechanisms remains underexplored. In this work, we classify neural operators into two types: (1) Spatial domain models that learn on grids and (2) Functional domain models that learn with function bases. We present several viewpoints based on this classification and focus on learning data-driven dynamics adhering to physical principles. Specifically, we provide a way to explain the prediction-making process of neural operators and show that neural operator can learn hidden physical patterns from data. However, this explanation method is limited to specific situations, highlighting the urgent need for generalizable explanation methods. Next, we show that a simple dual-space multi-scale model can achieve SOTA performance and we believe that dual-space multi-spatio-scale models hold significant potential to learn complex physics and require further investigation. Lastly, we discuss the critical need for principled frameworks to incorporate known physics into neural operators, enabling better generalization and uncovering more hidden physical phenomena.

Figures (9)

• Figure 1: Comparison of spatial dependencies of a wave propagation at different times: (a) Analytical spatial dependencies derived from the governing equations. (b) Spatial dependencies learned by a well-trained neural operator, demonstrating its ability to approximate the underlying spatial relationships. Spatial dependencies, defined in Appendix \ref{['append:spatial_dependencies_and_wave_analytical']}, describe which locations in the input domain have the greatest influence on the output, serving as a fundamental characteristic of many physical systems. For wave propagation, they reveal how disturbances at various input locations influence the resulting wave behavior, with isotropic patterns under certain conditions.
• Figure 2: Comparison of the analytical ERF function and that of the learned neural operators at various locations. The spatial domain models, CNO and GT-former, learn the wave patterns well. In contrast, the functional domain models, T1 and DeepONet, struggle to capture wave patterns. The hybrid model, FNO, is between these two models; it learns fairly well but introduces noise outside the arc region.
• Figure 3: Learned spatial dependencies on the incompressible Navier-Stokes equation reveal insightful patterns. The learned spatial dependencies for all spatial domain models (see Appendix \ref{['append: NS_equation']} for more results) align with expectations: as the vorticity constant $\nu$ decreases, local patterns become more significant, whereas global patterns gain importance with prolonged time scales.
• Figure 4: Enforcing physics priors into the network design constraints the learning space to be a subspace of operators that respect those physics priors. This reduces the need of learning data and enables better generalization, leading to more robust and interpretable results.
• Figure 5: In an operator learning task mapping atmospheric properties (e.g., pressure fields) to the temperature across the globe, rotation symmetries are desired. When you rotate the input atmospheric properties around the globe, the predicted temperature distribution should rotate accordingly.