Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving stiff dark matter equations via Jacobian Normalization with Physics-Informed Neural Networks

M. P. Bento, H. B. Câmara, J. R. Rocha, J. F. Seabra

TL;DR

This work proposes a simple, hyperparameter-free method to address stiffness by normalizing loss residuals with the Jacobian, and applies it to the stiff Boltzmann equations governing weakly interacting massive particle (WIMP) dark matter (DM).

Abstract

Stiff differential equations pose a major challenge for Physics-Informed Neural Networks (PINNs), often causing poor convergence. We propose a simple, hyperparameter-free method to address stiffness by normalizing loss residuals with the Jacobian. We provide theoretical indications that Jacobian-based normalization can improve gradient descent and validate it on benchmark stiff ordinary differential equations. We then apply it to a realistic system: the stiff Boltzmann equations (BEs) governing weakly interacting massive particle (WIMP) dark matter (DM). Our approach achieves higher accuracy than attention mechanisms previously proposed for handling stiffness, recovering the full solution where prior methods fail. This is further demonstrated in an inverse problem with a single experimental data point - the observed DM relic density - where our inverse PINNs correctly infer the cross section that solves the BEs in both Standard and alternative cosmologies.

Solving stiff dark matter equations via Jacobian Normalization with Physics-Informed Neural Networks

TL;DR

This work proposes a simple, hyperparameter-free method to address stiffness by normalizing loss residuals with the Jacobian, and applies it to the stiff Boltzmann equations governing weakly interacting massive particle (WIMP) dark matter (DM).

Abstract

Stiff differential equations pose a major challenge for Physics-Informed Neural Networks (PINNs), often causing poor convergence. We propose a simple, hyperparameter-free method to address stiffness by normalizing loss residuals with the Jacobian. We provide theoretical indications that Jacobian-based normalization can improve gradient descent and validate it on benchmark stiff ordinary differential equations. We then apply it to a realistic system: the stiff Boltzmann equations (BEs) governing weakly interacting massive particle (WIMP) dark matter (DM). Our approach achieves higher accuracy than attention mechanisms previously proposed for handling stiffness, recovering the full solution where prior methods fail. This is further demonstrated in an inverse problem with a single experimental data point - the observed DM relic density - where our inverse PINNs correctly infer the cross section that solves the BEs in both Standard and alternative cosmologies.
Paper Structure (10 sections, 44 equations, 6 figures, 1 table)

This paper contains 10 sections, 44 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Jacobian Normalization and Its Effect on PINN Convergence
  3. A benchmark on the simplest class of stiff ODEs
  4. Benchmarks on nonhomogeneous and nonlinear ODEs
  5. A Concrete Application: WIMP Dark Matter in Alternative Cosmologies
  6. Recasting the freeze-out Boltzmann equation
  7. Ablation test: Jacobian normalization versus vanilla PINN
  8. Jacobian normalization versus attention mechanisms
  9. Inverse problems: from the data to the possible physical models
  10. Concluding remarks

Figures (6)

  • Figure 1: PINN results for ODE \ref{['eq:case1']}, with normalized (unnormalized) residuals in green (red). Left: Maximum Hessian eigenvalue $\lambda_{\text{max}}$\ref{['eq:HessianGeneric']} in terms of the Jacobian $C$. The circles (crosses) indicate normalized (unnormalized) residuals. The linear fit slope gives the power-law scaling exponent of $\lambda_{\text{max}}$ with $C$. Right: Mean squared difference between PINN and FEM solutions \ref{['eq:epsilonMSEToy']} in terms of $C$.
  • Figure 2: PINN results for ODEs Eq. \ref{['eq:case2']} (left) and \ref{['eq:case3']} (right), with normalized [unnormalized] residuals in magenta [blue]. Top panels: Maximum Hessian eigenvalue $\lambda_{\text{max}}$\ref{['eq:HessianGeneric']} in terms of $C$. Bottom panels: Mean squared difference between PINN and FEM solutions $\overline{\varepsilon}$\ref{['eq:epsilonMSEToy']} in terms of $C$.
  • Figure 3: PINN $95\%$ confidence intervals for $\varepsilon_\text{rel}(z)$\ref{['eq:epsilonMSEzk']} with (magenta) and without (blue) Jacobian normalization, using linear (left) or negative sigmoid (right) output layers.
  • Figure 4: Solution of the freeze-out DM BE obtained with FEM (black dashed line) and various PINN approaches: vanilla (blue), Jacobian normalization (magenta), residual-based attention (green), and soft attention (brown). Left (Right): Results without (with) a negative sigmoid activation in the output layer. Top: Evolution of the WIMP DM particle yield. The horizontal dotted line satisfies the CDM abundance \ref{['eq:Oh2exp']}. Middle: Mean relative squared difference \ref{['eq:epsilonMSE']} between PINN and FEM during training. Bottom: Comparison of residual weights.
  • Figure 5: Diagrammatic representation of the inverse PINN structure modeling the WIMP DM BE for a given cosmological scenario (see text for details).
  • ...and 1 more figures