DAE-HardNet: A Physics Constrained Neural Network Enforcing Differential-Algebraic Hard Constraints

TL;DR

The paper addresses the gap in physics-informed learning where differential-algebraic constraints are only softly enforced. It introduces DAE-HardNet, a neural framework that uses a differentiable KKT-based projection layer to enforce hard DAEs and a Taylor-based neighborhood to couple state and derivatives, resulting in accurate predictions and near machine-precision constraint satisfaction across varied DAEs. The approach yields improved derivative information, robust parameter estimation, and competitive data losses compared with MLPs and PINNs, at the cost of increased training time. The work highlights practical pathways to apply hard-constrained learning to complex DAEs and suggests potential for bypassing the projection during fast inference when feasible.

Abstract

Traditional physics-informed neural networks (PINNs) do not always satisfy physics based constraints, especially when the constraints include differential operators. Rather, they minimize the constraint violations in a soft way. Strict satisfaction of differential-algebraic equations (DAEs) to embed domain knowledge and first-principles in data-driven models is generally challenging. This is because data-driven models consider the original functions to be black-box whose derivatives can only be obtained after evaluating the functions. We introduce DAE-HardNet, a physics-constrained (rather than simply physics-informed) neural network that learns both the functions and their derivatives simultaneously, while enforcing algebraic as well as differential constraints. This is done by projecting model predictions onto the constraint manifold using a differentiable projection layer. We apply DAE-HardNet to several systems and test problems governed by DAEs, including the dynamic Lotka-Volterra predator-prey system and transient heat conduction. We also show the ability of DAE-HardNet to estimate unknown parameters through a parameter estimation problem. Compared to multilayer perceptrons (MLPs) and PINNs, DAE-HardNet achieves orders of magnitude reduction in the physics loss while maintaining the prediction accuracy. It has the added benefits of learning the derivatives which improves the constrained learning of the backbone neural network prior to the projection layer. For specific problems, this suggests that the projection layer can be bypassed for faster inference. The current implementation and codes are available at https://github.com/SOULS-TAMU/DAE-HardNet.

Figures (14)

• Figure 1: Parity plots for predictions of functions and their derivatives.
• Figure 2: Neural network architecture of DAE-HardNet with exact DAEs satisfaction. Unconstrained outputs including the Lagrangian multipliers are evaluated using a standard neural net (learnable) and the derivative terms are calculated from the network through auto differentiation. The constrained outputs are calculated as the solution of the nonlinear system of equations (non-learnable) corresponding to the KKT relation of a distance minimization problem by considering the derivative terms as independent variables along with Taylor approximation.
• Figure 3: Projection of neural network outputs onto the constraint manifold. The target region is the actual solution of the problem. The difference between target region and projected region reduces with training as the derivative loss approaches zero.
• Figure 4: The pool of KKT systems created before model training due to different combination of the input features. During training BC: No, IC: No condition is imposed ensuring satisfaction of the governing equations during training for all points. Meanwhile during inference based on the input feature, corresponding KKT system is selected as the projection layer which satisfies the corresponding BC/IC if applicable.
• Figure 5: Learning curves for the system of ODEs in Example 1. Left: MSE data loss; Right: absolute constraint violation on the projected gradients over 5000 epochs.