HPX -- An open source C++ Standard Library for Parallelism and Concurrency

TL;DR

The paper argues that MPI+X is insufficient for modern heterogeneous HPC and proposes HPX, a C++ Standard–conforming AMT runtime with AGAS, futures, and a dataflow-oriented execution model. It details core runtime components, including thread management, global addressing, and active messaging, and showcases HPX’s applicability across diverse HPC domains with strong scalability, such as astrophysics and storm surge forecasting. The authors demonstrate substantial performance gains and parallel efficiency in real applications (e.g., $96.8\%$ on $643{,}280$ cores for STAR and near-peak single-node performance for LibGeoDecomp), highlighting HPX as a viable, open-source foundation for future HPC software stacks. The work emphasizes community-driven development and integration with open-source ecosystems, underscoring HPX’s potential to unify heterogeneous computing under a standards-compliant C++ runtime.

Abstract

To achieve scalability with today's heterogeneous HPC resources, we need a dramatic shift in our thinking; MPI+X is not enough. Asynchronous Many Task (AMT) runtime systems break down the global barriers imposed by the Bulk Synchronous Programming model. HPX is an open-source, C++ Standards compliant AMT runtime system that is developed by a diverse international community of collaborators called The Ste||ar Group. HPX provides features which allow application developers to naturally use key design patterns, such as overlapping communication and computation, decentralizing of control flow, oversubscribing execution resources and sending work to data instead of data to work. The Ste||ar Group comprises physicists, engineers, and computer scientists; men and women from many different institutions and affiliations, and over a dozen different countries. We are committed to advancing the development of scalable parallel applications by providing a platform for collaborating and exchanging ideas. In this paper, we give a detailed description of the features HPX provides and how they help achieve scalability and programmability, a list of applications of HPX including two large NSF funded collaborations (STORM, for storm surge forecasting; and STAR (OctoTiger) an astro-physics project which runs at 96.8% parallel efficiency on 643,280 cores), and we end with a description of how HPX and the Ste||ar Group fit into the open source community.

Figures (2)

• Figure 1: Overview of the HPX Runtime System components.
• Figure 2: Commits and contributors from the beginning of 2008 when HPX was hosted on github. The picures were taken from openhub.