Scalable Multilevel Monte Carlo Methods Exploiting Parallel Redistribution on Coarse Levels

TL;DR

The paper addresses MLMC scalability for PDEs with random coefficients by introducing level-dependent data redistribution in element-based AMGe to enable coarsening beyond the number of available cores. It preserves AMGe's coarse-space approximation properties and implements redistribution through parallel sparse-matrix operations, demonstrating improved coarse-level scalability. Numerical experiments on Darcy flow with log-normal permeability show significant MLMC CPU-time reductions and improved weak-scaling performance, validating the approach. The work suggests that extreme-scale simulations can benefit from coarse-level redistribution and outlines directions for extending these ideas to MLMC-based MCMC and multiple coarse problem copies.

Abstract

We study an element agglomeration coarsening strategy that requires data redistribution at coarse levels when the number of coarse elements becomes smaller than the used computational units (cores). The overall procedure generates coarse elements (general unstructured unions of fine grid elements) within the framework of element-based algebraic multigrid methods (or AMGe) studied previously. The AMGe generated coarse spaces have the ability to exhibit approximation properties of the same order as the fine-level ones since by construction they contain the piecewise polynomials of the same order as the fine level ones. These approximation properties are key for the successful use of AMGe in multilevel solvers for nonlinear partial differential equations as well as for multilevel Monte Carlo (MLMC) simulations. The ability to coarsen without being constrained by the number of available cores, as described in the present paper, allows to improve the scalability of these solvers as well as in the overall MLMC method. The paper illustrates this latter fact with detailed scalability study of MLMC simulations applied to model Darcy equations with a stochastic log-normal permeability field.

Figures (10)

• Figure 1: An example of mesh elements, the associated Raviart-Thomas dofs, true dofs, AEs, and relevant relation tables.
• Figure 2: An Illustration of (a) how "element_newelement", "AE_newelement", and "AE_element" connect element, "newelement", and AE, as in Algorithm \ref{['algorithm: parallel agglomeration']}; and (b) how "dof_newdof"$= ($"newdof_dof"$)^T$, $P^{new}$, and $P$ connect dof, "newdof", and coarse dofs, as in Algorithm \ref{['algorithm: data redistribution']}. Here, $\bullet$ and $\circ$ represent active and inactive cores respectively.
• Figure 3: Schematic of data redistribution on three levels for a unit cube spatial domain. Each color denotes an independent computational process appointed to a respective core, with (a) $\text{nc}=64$, (b) $\text{nc}=8$, and (c) $\text{nc}=1$ as the number of cores to which the mesh is redistributed for that level.
• Figure 4: (a) Weak scaling results comparing average walltime of a single Darcy simulations for five levels, where level $\ell=4$ scaling is displayed for simulations with and without redistribution. (b) Weak scaling efficiency for multiple levels as determined by computational time. While, at each level $\ell$, the non-redistribution and redistribution cases share approximately the same global number of elements (see Table \ref{['tab:ne']}), the act of redistributing the problem to fewer cores increases the local problem size. The simulation times are provided in Table \ref{['tab:ws-time']}.
• Figure 5: Comparison of the total walltime to build the Darcy AMGe hierarchy for the three Problems, split into the non-redistribution case and the redistribution case. Each bar is labeled with the walltime as well as the number of levels in the hierarchy, e.g., (6L) indicates six levels.

Theorems & Definitions (2)

• Remark 3.1: Comments on computational time
• Remark 5.1