SPIKE: Sparse Koopman Regularization for Physics-Informed Neural Networks

TL;DR

The paper tackles out-of-distribution generalization in physics-informed neural networks by introducing SPIKE, a framework that regularizes PINNs with a continuous-time Koopman generator. By learning a sparse, low-dimensional linear dynamics $dz/dt=Az$ in a combination of polynomial and latent observables, SPIKE achieves parsimonious, interpretable representations that improve temporal extrapolation and spatial generalization across a broad suite of PDEs and chaotic ODEs. Key contributions include continuous-time Koopman regularization with an optional matrix-exponential integrator for stiffness, explicit L1 sparsity to reveal active polynomial terms, and post-hoc coefficient recovery for inverse problems. Empirically, SPIKE yields substantial gains in open-domain and stiff systems, enables stable extrapolation, and reveals derivative-related structure in latent features, offering a practical, interpretable pathway to robust physics-based learning.

Abstract

Physics-Informed Neural Networks (PINNs) provide a mesh-free approach for solving differential equations by embedding physical constraints into neural network training. However, PINNs tend to overfit within the training domain, leading to poor generalization when extrapolating beyond trained spatiotemporal regions. This work presents SPIKE (Sparse Physics-Informed Koopman-Enhanced), a framework that regularizes PINNs with continuous-time Koopman operators to learn parsimonious dynamics representations. By enforcing linear dynamics $dz/dt = Az$ in a learned observable space, both PIKE (without explicit sparsity) and SPIKE (with L1 regularization on $A$) learn sparse generator matrices, embodying the parsimony principle that complex dynamics admit low-dimensional structure. Experiments across parabolic, hyperbolic, dispersive, and stiff PDEs, including fluid dynamics (Navier-Stokes) and chaotic ODEs (Lorenz), demonstrate consistent improvements in temporal extrapolation, spatial generalization, and long-term prediction accuracy. The continuous-time formulation with matrix exponential integration provides unconditional stability for stiff systems while avoiding diagonal dominance issues inherent in discrete-time Koopman operators.

Figures (5)

• Figure 1: PIKE/SPIKE architecture. A neural network maps coordinates $(x, y, t)$ to solution $u$, which is lifted to observables $g(u) = \psi_{\text{poly}}(u) \oplus \psi_{\text{latent}}(u)$. The Koopman operator $A$ governs linear dynamics $dz/dt = Az$. Three losses are minimized: physics residual $\mathcal{L}_{\text{physics}}$ via automatic differentiation, Koopman consistency $\mathcal{L}_{\text{koopman}}$, and L1 sparsity $\mathcal{L}_{\text{sparse}} = \|A\|_1$ (SPIKE only).
• Figure 2: Solution comparison for Burgers and Advection equations. Columns show temporal evolution from in-domain ($t \leq 1$) to OOD extrapolation ($t=1.5$). PINN exhibits oscillations near the Burgers shock and phase drift in Advection; PIKE/SPIKE variants maintain accuracy through Koopman regularization. Quantitative MSE values in Appendix \ref{['app:detailed_results']} (Tables \ref{['tab:integrator_ablation']}--\ref{['tab:integrator_ood_space']}).
• Figure 3: Lorenz system: PIKE/SPIKE achieve 184$\times$ longer valid prediction time than PINN ($\tau_{\text{PIKE}}/\tau_{\text{PINN}} = 11.02/0.06$, corresponding to $\approx$11 Lyapunov times vs $\approx$0.06).
• Figure 4: Navier-Stokes 2D lid-driven cavity flow ($Re=100$). PIKE-Euler achieves 2.3$\times$ lower in-domain MSE than PINN (1.62e-1 vs 3.74e-1). OOD region ($x > 0.7$) highlights spatial extrapolation capability within the bounded domain.
• Figure 5: Navier-Stokes 2D channel flow ($Re=100$): spatial OOD generalization. Training domain: $x \in [0,1]$; OOD region: $x \in [1,2]$ (downstream prediction within channel, $y \in [0,1]$). Solution error (shown): PIKE-EXPM achieves 5.5$\times$ lower error vs analytical Poiseuille flow than PINN (7.14e-2 vs 3.94e-1). Physics residual (Table \ref{['tab:integrator_2d_ood_space']}): PIKE-Euler achieves 23$\times$ lower residual (1.18e-1 vs 2.76e+0). The discrepancy reflects that minimizing PDE residual does not guarantee matching the true solution, particularly in extrapolation regions.$^{\dagger}$

Theorems & Definitions (22)

• Definition 1: Koopman Operator Family
• Proposition 1: Lie Operator Consistency
• Proposition 2: Finite-Dimensional Approximation
• Remark 1: Approximation Error Bounds
• Proposition 3: Sparsity and Polynomial Representation
• Remark 2: Lie Loss as Regularizer
• Proposition 4: Out-of-Distribution Generalization Bound
• Remark 3: Interpretation of Bound
• Definition 2: Augmented Embedding
• Lemma 1: Continuous Generator Advantage