Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems

Qifeng Hu, Shamsulhaq Basir, Inanc Senocak

TL;DR

The paper tackles the challenge of learning solutions to forward and inverse PDEs with multiple heterogeneous constraints by introducing CAPU, a per-constraint adaptive augmentation of the ALM. It couples constraint aggregation to reduce memory and improve training efficiency, and shows that a single Fourier feature mapping suffices to capture high-frequency, multi-scale solutions within PECANN. A time-windowing strategy enables stable long-time evolution without discrete time-stepping, expanding applicability to complex dynamics. Across heat transfer, Burgers’ rarefaction, Poisson, Helmholtz, and inverse heat-source problems, CAPU delivers competitive accuracy, robust constraint enforcement, and favorable training behavior compared to MPU/CPU baselines and baselines from COLLABORATIONS, illustrating the practical utility of constrained optimization principles in physics-informed learning.

Abstract

We present several key advances to the Physics and Equality Constrained Artificial Neural Networks (PECANN) framework, substantially improving its capacity to solve challenging partial differential equations (PDEs). Our enhancements broaden the framework's applicability and improve efficiency. First, we generalize the Augmented Lagrangian Method (ALM) to support multiple, independent penalty parameters for enforcing heterogeneous constraints. Second, we introduce a constraint aggregation technique to address inefficiencies associated with point-wise enforcement. Third, we incorporate a single Fourier feature mapping to capture highly oscillatory solutions with multi-scale features, where alternative methods often require multiple mappings or costlier architectures. Fourth, a novel time-windowing strategy enables seamless long-time evolution without relying on discrete time models. Fifth, and critically, we propose a conditionally adaptive penalty update (CAPU) strategy for ALM that accelerates the growth of Lagrange multipliers for constraints with larger violations, while enabling coordinated updates of multiple penalty parameters. CAPU accelerates the growth of Lagrange multipliers for selectively challenging constraints, enhancing constraint enforcement during training. We demonstrate the effectiveness of PECANN-CAPU across diverse problems, including the transonic rarefaction problem, reversible scalar advection by a vortex, high-wavenumber Helmholtz and Poisson's equations, and inverse heat source identification. The framework achieves competitive accuracy across all cases when compared with established methods and recent approaches based on Kolmogorov-Arnold networks. Collectively, these advances improve the robustness, computational efficiency, and applicability of PECANN to demanding problems in scientific computing.

Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems

TL;DR

The paper tackles the challenge of learning solutions to forward and inverse PDEs with multiple heterogeneous constraints by introducing CAPU, a per-constraint adaptive augmentation of the ALM. It couples constraint aggregation to reduce memory and improve training efficiency, and shows that a single Fourier feature mapping suffices to capture high-frequency, multi-scale solutions within PECANN. A time-windowing strategy enables stable long-time evolution without discrete time-stepping, expanding applicability to complex dynamics. Across heat transfer, Burgers’ rarefaction, Poisson, Helmholtz, and inverse heat-source problems, CAPU delivers competitive accuracy, robust constraint enforcement, and favorable training behavior compared to MPU/CPU baselines and baselines from COLLABORATIONS, illustrating the practical utility of constrained optimization principles in physics-informed learning.

Abstract

We present several key advances to the Physics and Equality Constrained Artificial Neural Networks (PECANN) framework, substantially improving its capacity to solve challenging partial differential equations (PDEs). Our enhancements broaden the framework's applicability and improve efficiency. First, we generalize the Augmented Lagrangian Method (ALM) to support multiple, independent penalty parameters for enforcing heterogeneous constraints. Second, we introduce a constraint aggregation technique to address inefficiencies associated with point-wise enforcement. Third, we incorporate a single Fourier feature mapping to capture highly oscillatory solutions with multi-scale features, where alternative methods often require multiple mappings or costlier architectures. Fourth, a novel time-windowing strategy enables seamless long-time evolution without relying on discrete time models. Fifth, and critically, we propose a conditionally adaptive penalty update (CAPU) strategy for ALM that accelerates the growth of Lagrange multipliers for constraints with larger violations, while enabling coordinated updates of multiple penalty parameters. CAPU accelerates the growth of Lagrange multipliers for selectively challenging constraints, enhancing constraint enforcement during training. We demonstrate the effectiveness of PECANN-CAPU across diverse problems, including the transonic rarefaction problem, reversible scalar advection by a vortex, high-wavenumber Helmholtz and Poisson's equations, and inverse heat source identification. The framework achieves competitive accuracy across all cases when compared with established methods and recent approaches based on Kolmogorov-Arnold networks. Collectively, these advances improve the robustness, computational efficiency, and applicability of PECANN to demanding problems in scientific computing.

Paper Structure

This paper contains 23 sections, 2 theorems, 57 equations, 22 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Technical Formulation
  3. Proposed Method: Conditionally Adaptive Augmented Lagrangian Method
  4. Constrained optimization formulation for solving forward and inverse problems
  5. Original PECANN formulation with point-wise constraints
  6. Revised PECANN formulation with constraint aggregation
  7. Extension to PDE-constrained inverse problems
  8. Role of convex distance functions in constraint enforcement and aggregation
  9. Integration of Fourier feature mappings into the PECANN framework
  10. Application to forward and inverse PDE problems
  11. Forward problem: transonic rarefaction in inviscid Burgers' equation
  12. Forward problem: 1D Poisson's equation
  13. Enhancements with a single Fourier feature mappings
  14. Forward problem: 2D Helmholtz equation with high-frequency and multi-scale solutions
  15. A much more challenging Helmholtz equation problem
  16. ...and 8 more sections

Key Result

Lemma 1

Let $\mu_i$ and $\lambda_i$ be the penalty parameter and Lagrange multiplier corresponding to the $i$th constraint $\mathcal{C}_i$ in the PECANN-CAPU algorithm. If the moving average of the squared constraint, denoted by $\bar{\nu_i}$, satisfies $\bar{\nu_i} \ll \epsilon$, where $\epsilon$ is a smal Consequently, when the constraint is sufficiently stabilized below $\sqrt{\epsilon}$, the moving av

Figures (22)

  • Figure 1: Heat transfer in a composite medium: Profiles of (a) predicted temperature and (b) predicted heat flux at $t = 1$, obtained using different penalty update strategies with the point-wise constraint formulation from Section \ref{['sec:constrained_optimization_formulation_pointwise_constraint']}.
  • Figure 2: Time-dependent heat conduction in a composite medium: Distribution of Lagrange multipliers obtained using the point-wise constraint formulation from Section \ref{['sec:constrained_optimization_formulation_pointwise_constraint']}. (a) MPU (Algorithm 1); (b) CPU (Algorithm 2); (c) CAPU (Algorithm 3, proposed).
  • Figure 3: Time-dependent heat conduction in a composite medium: Results are obtained using the constraint-aggregation formulation described in Section \ref{['sec:proposed_formulation']}. The prediction using CAPU with point-wise constraints is also included. Panels (a)-(c) show the contours of predicted temperature, the absolute error obtained with CAPU, and the evolution of the relative $l^2$ error $\mathcal{E}_r(\hat{u}, u)$, respectively, with the latter including comparisons among MPU, CPU, and CAPU (point-wise); panels (d)–(f) present the corresponding heat flux results.
  • Figure 4: Transonic rarefaction: Panels (a)–(c) show coarse-mesh results; panels (d)–(f) show fine-mesh counterparts. Specifically: (a, d) predictions at $t=0$; (b, e) predictions at $t=1$; (c, f) evolution of relative $l^2$ error norms. Curves represent mean predictions; shaded areas indicate standard deviation bands.
  • Figure 5: Transonic rarefaction: (a) exact solution, (b) prediction of the CAPU algorithm from the best fine-mesh trial, (c) the corresponding absolute point-wise error.
  • ...and 17 more figures

Theorems & Definitions (4)

  • Lemma 1: Boundedness of CAPU penalty parameters
  • proof
  • Theorem 1: Convergence of Lagrange multipliers in CAPU
  • proof