Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying constraint hierarchies in Bayesian PINNs via per-constraint Hessian decomposition

Filip Landgren

TL;DR

This paper tackles the problem of interpreting uncertainty in Bayesian physics-informed neural networks (B-PINNs), where physical constraints can cause overconfidence by constraining the solution manifold. It introduces a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into per-constraint contributions, enabling four metrics—Spectral Contribution, Alignment Score, Variance Attribution, and Condition-Number Ratio—to quantify how data, PDE residual, initial conditions, and boundary conditions shape the loss landscape. The approach is demonstrated on the Van der Pol oscillator, revealing that constraint influence is nonlinearly coupled and can reallocate across terms when weights are adjusted, sometimes leading to data- or IC-driven dominance despite nominal weights. This yields a diagnostic toolkit for diagnosing and potentially correcting curvature-driven miscalibration in B-PINNs, with implications for curvature-informed adaptive weighting and robust uncertainty quantification in physics-informed models. The framework relies on a Laplace approximation around the variational mean and uses matrix-free Hessian-vector products, Lanczos for top eigenpairs, and CG solves to obtain per-constraint curvature, without forming dense Hessians.

Abstract

Bayesian physics-informed neural networks (B-PINNs) merge data with governing equations to solve differential equations under uncertainty. However, interpreting uncertainty and overconfidence in B-PINNs requires care due to the poorly understood effects the physical constraints have on the network; overconfidence could reflect warranted precision, enforced by the constraints, rather than miscalibration. Motivated by the need to further clarify how individual physical constraints shape these networks, we introduce a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into contributions from each constraint and provides metrics to quantify their relative influence on the loss landscape. Applied to the Van der Pol equation, our method tracks how constraints sculpt the network's geometry and shows, directly through the Hessian, how changing a single loss weight non-trivially redistributes curvature and effective dominance across the others.

Quantifying constraint hierarchies in Bayesian PINNs via per-constraint Hessian decomposition

TL;DR

This paper tackles the problem of interpreting uncertainty in Bayesian physics-informed neural networks (B-PINNs), where physical constraints can cause overconfidence by constraining the solution manifold. It introduces a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into per-constraint contributions, enabling four metrics—Spectral Contribution, Alignment Score, Variance Attribution, and Condition-Number Ratio—to quantify how data, PDE residual, initial conditions, and boundary conditions shape the loss landscape. The approach is demonstrated on the Van der Pol oscillator, revealing that constraint influence is nonlinearly coupled and can reallocate across terms when weights are adjusted, sometimes leading to data- or IC-driven dominance despite nominal weights. This yields a diagnostic toolkit for diagnosing and potentially correcting curvature-driven miscalibration in B-PINNs, with implications for curvature-informed adaptive weighting and robust uncertainty quantification in physics-informed models. The framework relies on a Laplace approximation around the variational mean and uses matrix-free Hessian-vector products, Lanczos for top eigenpairs, and CG solves to obtain per-constraint curvature, without forming dense Hessians.

Abstract

Bayesian physics-informed neural networks (B-PINNs) merge data with governing equations to solve differential equations under uncertainty. However, interpreting uncertainty and overconfidence in B-PINNs requires care due to the poorly understood effects the physical constraints have on the network; overconfidence could reflect warranted precision, enforced by the constraints, rather than miscalibration. Motivated by the need to further clarify how individual physical constraints shape these networks, we introduce a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into contributions from each constraint and provides metrics to quantify their relative influence on the loss landscape. Applied to the Van der Pol equation, our method tracks how constraints sculpt the network's geometry and shows, directly through the Hessian, how changing a single loss weight non-trivially redistributes curvature and effective dominance across the others.

Paper Structure

This paper contains 4 sections, 3 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology and experimental setup
  3. Results
  4. Conclusions and outlook

Figures (2)

  • Figure 1: Top eigenspectra of the total Hessian across configurations (log scale). High-$\mu$ produces the sharpest curvature, reflecting stiff dynamics.
  • Figure 2: Constraint hierarchy ranks across configurations. Higher bars indicate stronger curvature contribution after combining SC, AS, VA, and CNR.