Implicit Neural Differential Model for Spatiotemporal Dynamics

TL;DR

This work tackles instability and error accumulation in long-horizon hybrid neural-physics models by introducing Im-PiNDiff, an implicit neural differential framework. It replaces explicit time stepping with implicit fixed-point updates and couples this with a hybrid gradient strategy that uses adjoint-state differentiation together with reverse-mode AD, complemented by checkpointing for memory efficiency. The approach is validated on canonical spatiotemporal PDEs, including advection-diffusion with steady and dynamic advection, Burgers' dynamics, and a multi-physics CVI problem, demonstrating improved stability, accuracy, and scalability as well as latent-physics inference from sparse data. The combination of implicit layers, CNF-based latent physics inference, and efficient gradient propagation marks a practical path toward robust, data-efficient hybrid modeling for complex scientific systems.”

Abstract

Hybrid neural-physics modeling frameworks through differentiable programming have emerged as powerful tools in scientific machine learning, enabling the integration of known physics with data-driven learning to improve prediction accuracy and generalizability. However, most existing hybrid frameworks rely on explicit recurrent formulations, which suffer from numerical instability and error accumulation during long-horizon forecasting. In this work, we introduce Im-PiNDiff, a novel implicit physics-integrated neural differentiable solver for stable and accurate modeling of spatiotemporal dynamics. Inspired by deep equilibrium models, Im-PiNDiff advances the state using implicit fixed-point layers, enabling robust long-term simulation while remaining fully end-to-end differentiable. To enable scalable training, we introduce a hybrid gradient propagation strategy that integrates adjoint-state methods with reverse-mode automatic differentiation. This approach eliminates the need to store intermediate solver states and decouples memory complexity from the number of solver iterations, significantly reducing training overhead. We further incorporate checkpointing techniques to manage memory in long-horizon rollouts. Numerical experiments on various spatiotemporal PDE systems, including advection-diffusion processes, Burgers' dynamics, and multi-physics chemical vapor infiltration processes, demonstrate that Im-PiNDiff achieves superior predictive performance, enhanced numerical stability, and substantial reductions in memory and runtime cost relative to explicit and naive implicit baselines. This work provides a principled, efficient, and scalable framework for hybrid neural-physics modeling.

Figures (15)

• Figure 1: Schematic of the Im-PiNDiff framework, where temporal states $\boldsymbol{\Phi}_t$ are advanced via implicit updates incorporating known and learned physics.
• Figure 2: Comparison between the schematics of PiNDiff and Im-PiNDiff layers.
• Figure 3: A schematic of forward and backward propagation through an implicit layer of the Im-PiNDiff model.
• Figure 4: Architecture of the conditional neural field (CNF) module for latent physics inference. A hypernetwork maps a contextual input vector $\mathbf{c}$ to a latent embedding $\mathbf{h}$, which is linearly projected to generate the weights $\boldsymbol{\theta}_b$ of a SIREN-based neural field.
• Figure 5: Forward prediction and inverse inference using Im-PiNDiff on the advection–diffusion problem with steady advection fields. The model is trained using only two snapshots ${\boldsymbol{\phi}t}{t=0, 0.05}$. Left: inferred steady advection fields $u_x(\mathbf{x})$ and $u_y(\mathbf{x})$ compared with ground truth. Right: predicted scalar field $\phi(\mathbf{x}, t)$ at future times, showing excellent agreement with ground truth across the forecast horizon.