Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learn Singularly Perturbed Solutions via Homotopy Dynamics

Chuqi Chen, Yahong Yang, Yang Xiang, Wenrui Hao

TL;DR

This work tackles the difficulty of solving singularly perturbed PDEs with neural networks by introducing Homotopy Dynamics, which traces a solution path as the perturbation parameter $\varepsilon$ is gradually varied from an easy regime to the target. The authors provide theoretical results showing how training difficulty scales with $\varepsilon$ via the neural-tangent-kernel framework and demonstrate convergence guarantees for the homotopy-path updates. They validate the method experimentally on the 2D Allen–Cahn equation, a high-dimensional Helmholtz equation, and operator-learning for Burgers’ equation, reporting faster convergence and improved accuracy relative to standard training. The approach offers a robust framework for extending neural PDE solvers to singularly perturbed problems and suggests promising directions for wider applicability and comparison with classical techniques.

Abstract

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Learn Singularly Perturbed Solutions via Homotopy Dynamics

TL;DR

This work tackles the difficulty of solving singularly perturbed PDEs with neural networks by introducing Homotopy Dynamics, which traces a solution path as the perturbation parameter is gradually varied from an easy regime to the target. The authors provide theoretical results showing how training difficulty scales with via the neural-tangent-kernel framework and demonstrate convergence guarantees for the homotopy-path updates. They validate the method experimentally on the 2D Allen–Cahn equation, a high-dimensional Helmholtz equation, and operator-learning for Burgers’ equation, reporting faster convergence and improved accuracy relative to standard training. The approach offers a robust framework for extending neural PDE solvers to singularly perturbed problems and suggests promising directions for wider applicability and comparison with classical techniques.

Abstract

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Paper Structure

This paper contains 29 sections, 8 theorems, 86 equations, 15 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Singularly perturbed Problems
  4. Neural Networks for Solving PDEs
  5. Challenges in Training Neural Networks
  6. Homotopy Dynamics
  7. Homotopy Path Tracking
  8. Incorporating Homotopy Dynamics into Neural Network Training
  9. Example: 1D Allen-Cahn steady-state equation.
  10. Theoretical analysis
  11. Challenges in Training Neural Network with Small Certain Parameters
  12. Convergence of Homotopy Dynamics
  13. Experiments
  14. 2D Allen Cahn Equation
  15. High-Dimensional Helmholtz Equation
  16. ...and 14 more sections

Key Result

Theorem 4.1

Suppose $\lambda_{\text{min}}(\boldsymbol{S}\boldsymbol{S}^{\top}) > 0$ and $\boldsymbol{D}_\varepsilon$ is non-singular, and let $\varepsilon \geq 0$ be a constant. Then, we have $\lambda_{\text{min}}(\boldsymbol{K}_{\varepsilon}) > 0$, and there exists $T > 0$ such that for all $t \in [0, T]$. Furthermore,

Figures (15)

  • Figure 1: Framework of homotopy dynamics for solving singularly perturbed problems.
  • Figure 2: Training curves for different values of $\varepsilon$ in solving the 1D Allen-Cahn steady-state equation. As $\varepsilon$ decreases, the training error increases, indicating that the training process becomes progressively more difficult.
  • Figure 3: Evolution of the Homotopy dynamics for steady state 1D Allen-Cahn equation. The L2RE for $\varepsilon=0.01$ is $8.08e-3$.
  • Figure 4: Largest eigenvalue of $\boldsymbol{D}_\varepsilon$\ref{['eq:discete_operator']} for different $\varepsilon$. A smaller $\varepsilon$ results in a smaller largest eigenvalue of \ref{['eq:discete_operator']}, leading to a slower convergence rate and increased difficulty in training.
  • Figure 5: 2D Allen Cahn Equaiton. (Top) Evolution of the Homotopy Dynamcis. (Bottom) Plot for Cross-section of $u(x,y)$ at $y = 0.5$ i.e., $u(x,y=0.5)$. The reference solution $u_{\infty}(x)$ represents the ground truth steady-state solution. The L2RE is $8.78e-3$. Number of residual points is $n_\textup{res} = 50\times50$.
  • ...and 10 more figures

Theorems & Definitions (15)

  • Theorem 4.1: Effectiveness of Training via the Eigenvalue of the Kernel
  • Remark 4.2
  • Theorem 4.3: Convergence of Homotopy Dynamics
  • Lemma 1.2: du2019gradient
  • Definition 1.3: vershynin2018high
  • Lemma 1.4
  • proof
  • Proposition 1.5: sub-exponential Bernstein's inequality vershynin2018high
  • Proposition 1.6
  • proof
  • ...and 5 more