Overlapping Schwarz Preconditioners for Randomized Neural Networks with Domain Decomposition

TL;DR

The paper addresses efficient PDE solving with randomized neural networks (RaNNs) connected through overlapping Schwarz domain decomposition. It introduces localized RaNN bases, a constraining operator to enforce boundary conditions without penalties, and PCA-based reduction to lower the parameter count, while constructing additive and restricted additive Schwarz preconditioners to accelerate the least-squares solves via CG/GMRES. Numerical experiments on multi-scale and time-dependent problems demonstrate substantial reductions in training time and improved conditioning, including a 3D example showing faster convergence than QR-based solves. The approach offers a mesh-free, scalable framework with potential for parallelization and extensions to multi-level Schwarz schemes.

Abstract

Randomized neural networks (RaNNs), in which hidden layers remain fixed after random initialization, provide an efficient alternative for parameter optimization compared to fully parameterized networks. In this paper, RaNNs are integrated with overlapping Schwarz domain decomposition in two (main) ways: first, to formulate the least-squares problem with localized basis functions, and second, to construct overlapping preconditioners for the resulting linear systems. In particular, neural networks are initialized randomly in each subdomain based on a uniform distribution and linked through a partition of unity, forming a global solution that approximates the solution of the partial differential equation. Boundary conditions are enforced through a constraining operator, eliminating the need for a penalty term to handle them. Principal component analysis (PCA) is employed to reduce the number of basis functions in each subdomain, yielding a linear system with a lower condition number. By constructing additive and restricted additive Schwarz preconditioners, the least-squares problem is solved efficiently using the Conjugate Gradient (CG) and Generalized Minimal Residual (GMRES) methods, respectively. Our numerical results demonstrate that the proposed approach significantly reduces computational time for multi-scale and time-dependent problems. Additionally, a three-dimensional problem is presented to demonstrate the efficiency of using the CG method with an AS preconditioner, compared to an QR decomposition, in solving the least-squares problem.

Figures (11)

• Figure 1: Example of using RaNNs with overlapping Schwarz DDM to solve PDEs in one dimension. The domain is divided into four subdomains, and RaNNs with one hidden layer and four hidden nodes are used for each subdomain. The window functions $w_j$, each with local support within $\Omega_i$, are displayed in the left figure. Exemplary local basis functions $\psi_j^k(x)$ for the second subdomain are shown in the right figure.
• Figure 2: Different values of $m$ are tested in (a) and (b), where the condition numbers are shown for $m = 16$ and $m =32$, respectively.
• Figure 3: The distribution of all 512 eigenvalues of the matrix $H^T H$ is shown.
• Figure 4: Network structure of the original RaNN $u_j:\mathbb{R}^{2}\rightarrow \mathbb{R}$ and new output $u_j^{p_j}$ after a PCA process, the solid blue line represents the parameters of the neural network that are randomly initialized and fixed thereafter, and the solid green line represents the PCA process, which involves a matrix multiplication with $V_{p_j} ^T$. The dotted red line indicates the parameters that need to be determined.
• Figure 5: The distribution of eigenvalues of the preconditioned matrix with different preconditioners using $4 \times 4$ domain decomposition in Example \ref{['ex1']}. The top figure compares the distribution of eigenvalues for the AS preconditioner against the distribution without preconditioning ordered in non-decreasing order, while the bottom figure illustrates the eigenvalue distributions for various preconditioners in the complex plane.

Theorems & Definitions (6)

• Remark 2.1
• Remark 3.1
• Remark 4.1
• Example 5.1
• Example 5.2
• Example 5.3