Towards Interpretable Deep Learning and Analysis of Dynamical Systems via the Discrete Empirical Interpolation Method

TL;DR

The paper addresses the interpretability and generalization limits of neural differential equation models for high-dimensional dynamics by introducing a differentiable adaptive DEIM framework that learns time-varying interpolation points with a Gumbel-Softmax mechanism, preserving $O(l)$ nonlinear-term evaluation. It demonstrates two main contributions: (i) end-to-end adaptive DEIM for Burgers’ equation that outperforms fixed-DEIM in capturing transient features, and (ii) DEIM-based analysis of a pre-trained NODE on a two-dimensional vortex-merging flow, showing that DEIM trajectories reveal physically meaningful structures and expose extrapolation weaknesses. The results establish DEIM as a dual tool for model reduction and diagnostic interpretation, with potential applications in data assimilation and sensor placement to improve predictive accuracy and stability of neural ODEs in complex flows.

Abstract

We present a differentiable framework that leverages the Discrete Empirical Interpolation Method (DEIM) for interpretable deep learning and dynamical system analysis. Although DEIM efficiently approximates nonlinear terms in projection-based reduced-order models (POD-ROM), its fixed interpolation points limit the adaptability to complex and time-varying dynamics. To address this limitation, we first develop a differentiable adaptive DEIM formulation for the one-dimensional viscous Burgers equation, which allows neural networks to dynamically select interpolation points in a computationally efficient and physically consistent manner. We then apply DEIM as an interpretable analysis tool for examining the learned dynamics of a pre-trained Neural Ordinary Differential Equation (NODE) on a two-dimensional vortex-merging problem. The DEIM trajectories reveal physically meaningful features in the learned dynamics of NODE and expose its limitations when extrapolating to unseen flow configurations. These findings demonstrate that DEIM can serve not only as a model reduction tool but also as a diagnostic framework for understanding and improving the generalization behavior of neural differential equation models.

Figures (12)

• Figure 1: (a) Velocity contour from the analytic solution; (b) Velocity contour from the FOM; (c) Squared error
• Figure 2: Effect of the temperature parameter $\tau$ on Gumbel–Softmax samples. Lower values of $\tau$ produce sharper, nearly one-hot samples, while higher values lead to smoother and more uniform distributions.
• Figure 3: Velocity prediction contours obtained from the original DEIM and the trained adaptive DEIM model, along with the corresponding squared error contours for each model
• Figure 4: Mean squared error (MSE) at each time step for the original DEIM and the trained adaptive DEIM model
• Figure 5: Trajectories of sampling points for the original DEIM and the trained adaptive DEIM model