Architecture-Optimization Co-Design for Physics-Informed Neural Networks Via Attentive Representations and Conflict-Resolved Gradients

TL;DR

This work addresses the dual challenges of limited representational capacity and gradient conflicts in physics-informed neural networks (PINNs) by proposing an architecture–optimization co-design. It introduces Layer-wise Dynamic Attention (LDA) to re-encode inputs at every layer and a conflict-aware gradient update (PCGrad-style) to mitigate destructive gradient interferences, culminating in the Architecture–Conflict-Resolved PINN (ACR-PINN). Across Burgers, Helmholtz, Klein–Gordon, and lid-driven cavity benchmarks, ACR-PINN achieves faster convergence and substantially lower relative $L_2$ and $L_infty$ errors than standard PINNs, demonstrating the value of jointly optimizing representation and gradient interactions for multiscale and oscillatory PDEs. The approach holds promise for more robust, high-accuracy PINN solvers in complex multi-physics settings.

Abstract

Physics-Informed Neural Networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs) by embedding governing physical laws into neural network training. In practice, however, their performance is often hindered by limited representational capacity and optimization difficulties caused by competing physical constraints and conflicting gradients. In this work, we study PINN training from a unified architecture-optimization perspective. We first propose a layer-wise dynamic attention mechanism to enhance representational flexibility, resulting in the Layer-wise Dynamic Attention PINN (LDA-PINN). We then reformulate PINN training as a multi-task learning problem and introduce a conflict-resolved gradient update strategy to alleviate gradient interference, leading to the Gradient-Conflict-Resolved PINN (GC-PINN). By integrating these two components, we develop the Architecture-Conflict-Resolved PINN (ACR-PINN), which combines attentive representations with conflict-aware optimization while preserving the standard PINN loss formulation. Extensive experiments on benchmark PDEs, including the Burgers, Helmholtz, Klein-Gordon, and lid-driven cavity flow problems, demonstrate that ACR-PINN achieves faster convergence and significantly lower relative $L_2$ and $L_\infty$ errors than standard PINNs. These results highlight the effectiveness of architecture-optimization co-design for improving the robustness and accuracy of PINN-based solvers.

Figures (22)

• Figure 1: Overall framework of the standard PINN.
• Figure 2: Overall architecture of the LDA module.
• Figure 3: Overall architecture of the proposed LDA-PINN.
• Figure 4: Geometric illustration of cooperative and conflicting gradients in multi-task PINN optimization.
• Figure 5: Geometric illustration of the two PCGrad projection directions under gradient conflict.