Hard-Constrained Neural Networks with Physics-Embedded Architecture for Residual Dynamics Learning and Invariant Enforcement in Cyber-Physical Systems

TL;DR

The paper tackles learning for cyber-physical systems described by differential equations with unknown residual dynamics and algebraic invariants. It introduces HRPINN, a hard-constraint recurrent architecture that embeds known physics and learns only residual dynamics, and its extension PHRPINN, which adds a differentiable projection to strictly enforce algebraic invariants. A key theoretical result proves representational equivalence between HRPINN and standard PINNs, ensuring expressive parity while offering optimization and data-efficiency gains; the authors also provide a seven-step construction method and discuss gradient strategies. Empirical validation on a real-world battery DAE and standard constrained benchmarks demonstrates improved data efficiency, physical consistency, and robust optimization for HRPINN, with PHRPINN delivering strict invariant enforcement at the cost of computational overhead. The work offers a principled pathway to dependable digital twins for safety-critical CPS, balancing physical fidelity with learning capacity and deployment guarantees.

Abstract

This paper presents a framework for physics-informed learning in complex cyber-physical systems governed by differential equations with both unknown dynamics and algebraic invariants. First, we formalize the Hybrid Recurrent Physics-Informed Neural Network (HRPINN), a general-purpose architecture that embeds known physics as a hard structural constraint within a recurrent integrator to learn only residual dynamics. Second, we introduce the Projected HRPINN (PHRPINN), a novel extension that integrates a predict-project mechanism to strictly enforce algebraic invariants by design. The framework is supported by a theoretical analysis of its representational capacity. We validate HRPINN on a real-world battery prognostics DAE and evaluate PHRPINN on a suite of standard constrained benchmarks. The results demonstrate the framework's potential for achieving high accuracy and data efficiency, while also highlighting critical trade-offs between physical consistency, computational cost, and numerical stability, providing practical guidance for its deployment.

Figures (3)

• Figure 1: A comparative illustration of the PIML architectures discussed. NODE learns the entire dynamics $\hat{f}_{\theta}$ as a black-box. PNODE augments this with a hard-constraint projection step to enforce invariants $g(x)=0$. PINN approximates the solution trajectory $\hat{x}_{\theta}(t)$ directly and enforces physics via a soft penalty in the loss function. In contrast, our proposed HRPINN hard-codes the known physics $f_{phys}$ and uses a neural network to learn only the residual dynamics $\hat{f}_{\theta}$. Our PHRPINN extends this residual-learning approach by integrating a final projection step, strictly enforcing algebraic invariants $g(x)=0$ by design.
• Figure 2: Generalization learning curves for the "RobotArm" system (Conjecture 2). PHRPINN (blue) achieves significantly lower MSE than PINN (red), even with a fraction of the training data. The fluctuations in the PHRPINN curve, while appearing large on the logarithmic scale, are characteristic of the training process, arising from the stochastic nature of mini-batch sampling and optimizer dynamics.
• Figure 3: Comprehensive generalization learning curves for all six benchmark systems, supporting the discussion for Conjecture 2. Each subplot compares the Mean Squared Error (log scale) of PHRPINN (blue) and PINN (red) as the percentage of available training data increases. These results highlight the system-dependent nature of the generalization advantage conferred by PHRPINN's strong inductive bias, showing strong outperformance on "RobotArm" but underperformance on simpler systems like "MassSpring".

Theorems & Definitions (5)

• Definition 1: Orthogonal Manifold Projection
• Theorem 2: Representational Equivalence
• Conjecture 3: Improved Optimization Conditioning
• Conjecture 4: Superior Generalization
• Lemma 5: Differentiability of the orthogonal projection