Solving Differential Equations using Physics-Informed Deep Equilibrium Models

TL;DR

This work addresses solving initial-value problems (IVPs) for ODEs under limited data by marrying physics-informed training with Deep Equilibrium Models (DEQs) to form Physics-Informed Deep Equilibrium Models (PIDEQs). PIDEQs enforce dynamics via a physics-informed loss and stabilize training with a Jacobian Frobenius-norm regularization, while solving the equilibrium with implicit differentiation. Empirical evaluation on the Van der Pol oscillator shows that, although PINNs can achieve lower error in this setting, PIDEQs achieve competitive accuracy with potentially faster convergence and offer a pathway to more complex systems, including higher-order ODEs and PDEs, through improved backward passes. The findings highlight a promising direction for integrating physics-based modeling with implicit-depth architectures, while pointing to future work on more robust backward schemes and broader problem classes.

Abstract

This paper introduces Physics-Informed Deep Equilibrium Models (PIDEQs) for solving initial value problems (IVPs) of ordinary differential equations (ODEs). Leveraging recent advancements in deep equilibrium models (DEQs) and physics-informed neural networks (PINNs), PIDEQs combine the implicit output representation of DEQs with physics-informed training techniques. We validate PIDEQs using the Van der Pol oscillator as a benchmark problem, demonstrating their efficiency and effectiveness in solving IVPs. Our analysis includes key hyperparameter considerations for optimizing PIDEQ performance. By bridging deep learning and physics-based modeling, this work advances computational techniques for solving IVPs, with implications for scientific computing and engineering applications.

Figures (8)

• Figure 1: Illustration of the recursion within the equilibrium equation that defines the DEQs.
• Figure 2: Solution for the Van der Pol oscillator with initial condition $y_1(0)=0,\,y_2(0)=0.1$ and $\mu=1$. The trajectory of the numerical solution is shown in a state-space plot, with $y_2$ in the vertical axis and $y_1$ in the horizontal axis.
• Figure 3: The learning curve for the baseline models trained on the IVP of the Van der Pol oscillator. Solid lines are mean values ($n=5$), and shaded regions represent minimum and maximum values. For a better visualization, a moving average of 100 epochs was taken.
• Figure 4: Learning curve of the PIDEQs trained with a varying number of states. The model with 80 hidden states is the same as the baseline PIDEQ from Fig. \ref{['fig:baseline-iae']}. Solid lines are mean values ($n=5$). For a better visualization, a moving average of 100 epochs was taken.
• Figure 5: Learning curve of PIDEQs with different $\kappa$ values. Only one model was trained with $\kappa=0$ because the training took over 30 times longer. For all other scenarios, we present the mean over five runs (parameter initializations). For a better visualization, a moving average of 100 epochs was taken.