Scalable overset computation between a forest-of-octrees- and an arbitrary distributed parallel mesh

TL;DR

An algorithm is introduced that performs a one-directional mesh overset of a parallel forest of octrees with another distributed mesh of unrelated partition that can be generalized, for example to load balancing of the overset, or adaptive refinement of the meshes around their intersection area.

Abstract

We introduce an algorithm that performs a one-directional mesh overset of a parallel forest of octrees with another distributed mesh of unrelated partition. The forest mesh consists of several adaptively refined octrees. Individual smooth mappings for every tree allow to represent a broad range of geometric domains. The other mesh is generic and defines a distributed set of query points, e.g. stemming from a quadrature rule applied to each cell. We face the problem of finding data for all queries in the remote forest. The forest is partitioned according to its natural Morton ordering. Thus, the partition boundaries can be encoded globally with one Morton index per process, which allows for precise, communication-free searching of the queries in the partition geometry. This is necessary to organize non-blocking communication of the queries to the relevant processes only. In a subsequent local search of the forest, we process the incoming queries and return the data of interest to each query's origin. The algorithm can be generalized, for example to load balancing of the overset, or adaptive refinement of the meshes around their intersection area. In 2D and 3D example scenarios we demonstrate the algorithm's performance and scalability to 12,288 processes.

Figures (21)

• Figure 1: Example of a forest of $K=2$ quadtrees (top) and the corresponding mesh in the geometric domain (bottom left). The black arrow shows the ordering of the mesh quadrants according to the Morton space-filling curve. In this example the curves of both trees have the same orientation, but this is not necessarily so. The mesh is partitioned among three processes represented by colors and divided by black dashes. The mesh on the bottom right is the coarsest mesh suited to represent the partition boundaries exactly. It can be deduced communication-free from the global first positions (small, black cells) and is the basis of the partition search, even without ever being constructed explicitly.
• Figure 2: Local search of three points (blue) on process $2$ in the example forest from Figure \ref{['fig:octree_partition']}, shown for the tree $k_1$ (top) and the corresponding part of the mesh (bottom). Green cells containing a check mark indicate at least one point match in the given quadrant. Red cells containing a cross indicate no match. A gray cell indicates that the quadrant was not part of the search, either because it was visited before (hatched cells) or an ancestor cell yielded no match (plain cells). Cells in the tree belonging to a different process are white. The four meshes on the bottom describe the execution of the search by increasing levels of the tree.
• Figure 3: Partition search of five points (blue) in the example forest from Figure \ref{['fig:octree_partition']}, shown for both the trees (top) and the corresponding part of the mesh (bottom). Green cells containing a check mark indicate at least one point match in the given quadrant. Red cells containing a cross indicate no match. A gray cell indicates that the quadrant was not part of the search, either because it was visited before (hatched cells) or an ancestor cell yielded no match (plain cells). The global first positions (small, black cells) mark partition boundaries and thus trigger refinement of the search tree. Cells whose parent belongs to exactly one process cannot occur in the partition search and are marked in white. The search can be imagined to operate on the partition-induced tree from Figure \ref{['fig:octree_partition']}. The three meshes on the bottom describe the behavior of the search on the different levels of the tree. Level $3$ was omitted since the search does not touch it.
• Figure 4: Flowchart of the mesh overset algorithm. The chart shows the local computations of rank $0$ of both the consumer (left, orange) and producer (right, blue) communicator as well as the communication they participate in (green, dashed lines). One query point is highlighted as a light blue circle during the whole algorithm. The numbers in orange circles offer a proposed reading order of the flow chart. The various steps are exposed in detail throughout Section \ref{['sec:overset_algorithm']}.
• Figure 5: Example for the geometric setup of the mesh overset. The consumer side is displayed in orange and the producer side in blue. The consumer mesh is unstructured and consists of triangles, quadrilaterals and a pentagon. We have one query point for every cell center. The producer mesh is a forest of two octrees. Both (square) reference domains are mapped to physical space by a mapping $\psi_k$, respectively. The resulting geometric domain is connected, since the trees intersect precisely at the edge $\psi_0 \left(\Gamma_1\right) = \psi_1 \left(\Gamma_2\right)$. Note that the parallel partition and distributed nature of the two respective meshes is not shown in this display.