Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive-Growth Randomized Neural Networks for PDEs: Algorithms and Numerical Analysis

Haoning Dang, Fei Wang, Song Jiang

TL;DR

This paper tackles the data- and computation-intensive challenge of solving PDEs with mesh-free randomized neural networks by proposing AG-RaNN, which adaptively grows width and depth guided by residuals and frequency information. The authors introduce a constructive, residual-driven workflow: frequency-based parameter initialization, neuron growth, layer growth, and domain splitting, all while maintaining a mostly fixed-parameter RaNN backbone. They develop a unified graph-norm-based error analysis, decomposing the total error into approximation, statistical, and optimization components, and prove convergence under mild assumptions for single-hidden-layer RaNN solvers. Extensive numerical experiments across Poisson, Burgers, and Allen–Cahn-type problems demonstrate that AG-RaNN can achieve high accuracy with substantially fewer degrees of freedom than traditional methods, particularly for problems with sharp gradients or discontinuities. The work offers a practical, theory-backed framework for adaptive, mesh-free PDE solvers with potential extensions to higher-dimensional and multi-physics problems.

Abstract

Randomized neural network (RaNN) methods have been proposed for solving various partial differential equations (PDEs), demonstrating high accuracy and efficiency. However, initializing the fixed parameters remains challenging. Additionally, RaNNs often struggle to approximate PDE solutions with sharp gradients or discontinuities when using smooth activations and shallow architectures. In this paper, we propose an Adaptive-Growth Randomized Neural Network (AG-RaNN) to address these challenges. We first design a frequency-based initialization for a shallow RaNN. Using the residual as an error indicator, we then adaptively grow the network in width (neuron growth) and depth (layer growth) to improve the accuracy of the numerical solution. The weights and biases of new neurons are constructed rather than trained, which enhances the approximation power without additional nonlinear optimization. To handle discontinuities, we further introduce a domain splitting strategy. We also establish a unified error analysis covering approximation, statistical, and optimization errors. Extensive numerical experiments demonstrate the efficiency and accuracy of AG-RaNN.

Adaptive-Growth Randomized Neural Networks for PDEs: Algorithms and Numerical Analysis

TL;DR

This paper tackles the data- and computation-intensive challenge of solving PDEs with mesh-free randomized neural networks by proposing AG-RaNN, which adaptively grows width and depth guided by residuals and frequency information. The authors introduce a constructive, residual-driven workflow: frequency-based parameter initialization, neuron growth, layer growth, and domain splitting, all while maintaining a mostly fixed-parameter RaNN backbone. They develop a unified graph-norm-based error analysis, decomposing the total error into approximation, statistical, and optimization components, and prove convergence under mild assumptions for single-hidden-layer RaNN solvers. Extensive numerical experiments across Poisson, Burgers, and Allen–Cahn-type problems demonstrate that AG-RaNN can achieve high accuracy with substantially fewer degrees of freedom than traditional methods, particularly for problems with sharp gradients or discontinuities. The work offers a practical, theory-backed framework for adaptive, mesh-free PDE solvers with potential extensions to higher-dimensional and multi-physics problems.

Abstract

Randomized neural network (RaNN) methods have been proposed for solving various partial differential equations (PDEs), demonstrating high accuracy and efficiency. However, initializing the fixed parameters remains challenging. Additionally, RaNNs often struggle to approximate PDE solutions with sharp gradients or discontinuities when using smooth activations and shallow architectures. In this paper, we propose an Adaptive-Growth Randomized Neural Network (AG-RaNN) to address these challenges. We first design a frequency-based initialization for a shallow RaNN. Using the residual as an error indicator, we then adaptively grow the network in width (neuron growth) and depth (layer growth) to improve the accuracy of the numerical solution. The weights and biases of new neurons are constructed rather than trained, which enhances the approximation power without additional nonlinear optimization. To handle discontinuities, we further introduce a domain splitting strategy. We also establish a unified error analysis covering approximation, statistical, and optimization errors. Extensive numerical experiments demonstrate the efficiency and accuracy of AG-RaNN.
Paper Structure (19 sections, 10 theorems, 141 equations, 23 figures, 13 tables)

This paper contains 19 sections, 10 theorems, 141 equations, 23 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Neural Networks
  3. Fully Connected Feedforward Neural Networks
  4. Shortcut Connections
  5. Randomized Neural Networks
  6. Adaptive-Growth RaNN
  7. Frequency-Based Parameter Initialization Strategy
  8. Neuron Growth Strategy
  9. Layer Growth Strategy
  10. Domain Splitting Strategy
  11. Numerical Analysis
  12. Population and empirical losses.
  13. Approximation Error
  14. Statistical Component
  15. A convergence result
  16. ...and 4 more sections

Key Result

Lemma 4.5

For a bounded Lipschitz domain $D\subset\mathbb{R}^d$, there exists a bounded linear extension operator $E:H^{s+\varepsilon_C}(D)\to H^{s+\varepsilon_C}(\mathbb{R}^d)$ such that $(Eu^*)|_D=u^*$ and $\|Eu^*\|_{s+\varepsilon_C,\mathbb{R}^d}\,\le\,C_E(\varepsilon_C,D,d)\,\|u^*\|_{s+\varepsilon_C,D}$.

Figures (23)

  • Figure 1: An RaNN with shortcut connections (solid lines: fixed; dotted lines: trainable). Configuration: $(d,m_1,m_2,m_3,m_4)=(2,5,3,3,1)$.
  • Figure 2: Flowchart of the AG-RaNN algorithm.
  • Figure 3: Initial RaNN, neuron growth, and RaNN with a shortcut connection.
  • Figure 4: Randomized neural network with localized basis functions.
  • Figure 5: Best result obtained by the manually tuned RaNN in Case 1 of Example \ref{['ex:Poisson']}.
  • ...and 18 more figures

Theorems & Definitions (23)

  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Remark 3.6
  • Remark 4.2: Typical choices of $(V,Y,Z)$ for PDE models
  • Remark 4.4
  • Lemma 4.5: Extension, Adams1975Sobolev
  • Lemma 4.6: Bandlimit error
  • ...and 13 more