Efficient Shallow Ritz Method For 1D Diffusion-Reaction Problems

TL;DR

This work develops an efficient shallow Ritz discretization for one-dimensional diffusion-reaction problems by coupling a shallow ReLU NN with a damped block Newton (dBN) method, enabling near-optimal approximation orders even for non-smooth cases. Key contributions include a rigorous optimality framework, an $O(n)$-cost inversion strategy for the dense, ill-conditioned mass matrix via algebraic and geometric factorizations, and a reduced, invertible nonlinear system that sustains Newton updates despite Hessian singularities. The authors extend the framework to least-squares problems, demonstrate adaptive breakpoint placement via ANE/AdBN to improve convergence, and validate the approach with numerical experiments showing competitive or superior performance to L-BFGS and adaptive FEM, particularly for singularly perturbed problems. Overall, the method offers a scalable, accurate, and adaptable solver for 1D problems and provides a pathway to higher-dimensional extensions and rigorous convergence analyses in future work.

Abstract

This paper studies the shallow Ritz method for solving one-dimensional diffusion-reaction problems. The method is capable of improving the order of approximation for non-smooth problems. By following a similar approach to the one presented in [9], we present a damped block Newton (dBN) method to achieve nearly optimal order of approximation. The dBN method optimizes the Ritz functional by alternating between the linear and non-linear parameters of the shallow ReLU neural network (NN). For diffusion-reaction problems, new difficulties arise: (1) for the linear parameters, the mass matrix is dense and even more ill-conditioned than the stiffness matrix, and (2) for the non-linear parameters, the Hessian matrix is dense and may be singular. This paper addresses these challenges, resulting in a dBN method with computational cost of ${\cal O}(n)$. The ideas presented for diffusion-reaction problems can also be applied to least-squares approximation problems. For both applications, starting with the non-linear parameters as a uniform partition, numerical experiments show that the dBN method moves the mesh points to nearly optimal locations.

Figures (6)

• Figure 1: Comparison between L-BFGS and dBN for approximating function \ref{['Example2eq']}.
• Figure 2: (a) initial NN model with 15 uniform breakpoints, $J(u_n) = 5.64 \times 10^{-5}$, (b) optimized NN model with 15 breakpoints using dBN, 1000 iterations, $J(u_n) = 1.20 \times 10^{-7}$.
• Figure 3: Comparison between L-BFGS and dBN for approximating function \ref{['Example1eq']}.
• Figure 4: (a) initial NN model with 22 uniform breakpoints, $e_n = 0.228$, (b) optimized NN model with 22 breakpoints, 500 iterations, $e_n = 0.092$, (c) adaptive approximation ($n = 8, 11, 14, 18, 22$), $e_n = 0.083$.
• Figure 5: For $\nu = \varepsilon^2 = 10^{-4}$: (a) initial NN model with 20 uniform breakpoints, $e_n = 0.935$, (b) optimized NN model with 20 breakpoints, 200 iterations, $e_n =0.130$.

Theorems & Definitions (31)

• Proposition 2.1
• Proof 1
• Remark 3.1
• Lemma 4.1
• Proof 2
• Lemma 4.2
• Proof 3
• Lemma 4.3
• Proof 4
• Lemma 4.4