Layout-Agnostic MPI Abstraction for Distributed Computing in Modern C++
Jiří Klepl, Martin Kruliš, Matyáš Brabec
TL;DR
The paper addresses the rigidity of MPI's legacy C interface by introducing Noarr-MPI, a layout- and traversal-agnostic MPI abstraction built on the Noarr framework. It presents a recursion-based type transformation that auto-generates MPI datatypes from Noarr layouts and introduces an MPITraverser that unifies iteration with communication. The authors demonstrate the approach with a distributed GEMM kernel, showing performance on par with state-of-the-art C++ MPI bindings while enabling more flexible data layouts and easier integration with accelerator backends. The work suggests significant practical impact by reducing datatype boilerplate and enabling seamless layout transformations without data movement overhead, potentially simplifying the development of high-performance distributed applications. Overall, Noarr-MPI advances layout-aware MPI programming and demonstrates effective collaboration with Noarr traversers for broader HPC workflows.
Abstract
Message Passing Interface (MPI) has been a well-established technology in the domain of distributed high-performance computing for several decades. However, one of its greatest drawbacks is a rather ancient pure-C interface. It lacks many useful features of modern languages (namely C++), like basic type-checking or support for generic code design. In this paper, we propose a novel abstraction for MPI, which we implemented as an extension of the C++ Noarr library. It follows Noarr paradigms (first-class layout and traversal abstraction) and offers layout-agnostic design of MPI applications. We also implemented a layout-agnostic distributed GEMM kernel as a case study to demonstrate the usability and syntax of the proposed abstraction. We show that the abstraction achieves performance comparable to the state-of-the-art MPI C++ bindings while allowing for a more flexible design of distributed applications.