Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Performance-Productivity Trade-offs in AMT Runtimes: A Task Bench Study of Itoyori, ItoyoriFBC, HPX, and MPI

Torben R. Lahnor, Mia Reitz, Jonas Posner, Patrick Diehl

TL;DR

The paper addresses how to fairly compare asynchronous many-task (AMT) runtimes against MPI on HPC clusters by integrating Itoyori and ItoyoriFBC into the Task Bench framework. It conducts a comprehensive performance and productivity study across compute-bound, weak-scaling, imbalance, and communication-heavy workloads, using Application Efficiency and METG as core metrics. Key contributions include the first Task Bench implementations for Itoyori and ItoyoriFBC, improvements to the HPX Task Bench implementation, and a nuanced set of findings: MPI is best for regular workloads, Itoyori excels in communication-intensive scenarios with strong productivity benefits, HPX offers robust load-balancing performance under imbalance, and ItoyoriFBC introduces DAG expressiveness at an overhead cost. The work demonstrates meaningful trade-offs among runtime models and shows that Task Bench enables apples-to-apples comparisons, guiding researchers and practitioners in selecting runtimes for irregular and communication-rich HPC workloads.

Abstract

Asynchronous Many-Task (AMT) runtimes offer a productive alternative to the Message Passing Interface (MPI). However, the diverse AMT landscape makes fair comparisons challenging. Task Bench, proposed by Slaughter et al., addresses this challenge through a parameterized framework for evaluating parallel programming systems. This work integrates two recent cluster AMTs, Itoyori and ItoyoriFBC, into Task Bench for comprehensive evaluation against MPI and HPX. Itoyori employs a Partitioned Global Address Space (PGAS) model with RDMA-based work stealing, while ItoyoriFBC extends it with futurebased synchronization. We evaluate these systems in terms of both performance and programmer productivity. Performance is assessed across various configurations, including compute-bound kernels, weak scaling, and both imbalanced and communication-intensive patterns. Performance is quantified using application efficiency, i.e., the percentage of maximum performance achieved, and the Minimum Effective Task Granularity (METG), i.e., the smallest task duration before runtime overheads dominate. Programmer productivity is quantified using Lines of Code (LOC) and the Number of Library Constructs (NLC). Our results reveal distinct trade-offs. MPI achieves the highest efficiency for regular, communication-light workloads but requires verbose, lowlevel code. HPX maintains stable efficiency under load imbalance across varying node counts, yet ranks last in productivity metrics, demonstrating that AMTs do not inherently guarantee improved productivity over MPI. Itoyori achieves the highest efficiency in communication-intensive configurations while leading in programmer productivity. ItoyoriFBC exhibits slightly lower efficiency than Itoyori, though its future-based synchronization offers potential for expressing irregular workloads.

Exploring Performance-Productivity Trade-offs in AMT Runtimes: A Task Bench Study of Itoyori, ItoyoriFBC, HPX, and MPI

TL;DR

The paper addresses how to fairly compare asynchronous many-task (AMT) runtimes against MPI on HPC clusters by integrating Itoyori and ItoyoriFBC into the Task Bench framework. It conducts a comprehensive performance and productivity study across compute-bound, weak-scaling, imbalance, and communication-heavy workloads, using Application Efficiency and METG as core metrics. Key contributions include the first Task Bench implementations for Itoyori and ItoyoriFBC, improvements to the HPX Task Bench implementation, and a nuanced set of findings: MPI is best for regular workloads, Itoyori excels in communication-intensive scenarios with strong productivity benefits, HPX offers robust load-balancing performance under imbalance, and ItoyoriFBC introduces DAG expressiveness at an overhead cost. The work demonstrates meaningful trade-offs among runtime models and shows that Task Bench enables apples-to-apples comparisons, guiding researchers and practitioners in selecting runtimes for irregular and communication-rich HPC workloads.

Abstract

Asynchronous Many-Task (AMT) runtimes offer a productive alternative to the Message Passing Interface (MPI). However, the diverse AMT landscape makes fair comparisons challenging. Task Bench, proposed by Slaughter et al., addresses this challenge through a parameterized framework for evaluating parallel programming systems. This work integrates two recent cluster AMTs, Itoyori and ItoyoriFBC, into Task Bench for comprehensive evaluation against MPI and HPX. Itoyori employs a Partitioned Global Address Space (PGAS) model with RDMA-based work stealing, while ItoyoriFBC extends it with futurebased synchronization. We evaluate these systems in terms of both performance and programmer productivity. Performance is assessed across various configurations, including compute-bound kernels, weak scaling, and both imbalanced and communication-intensive patterns. Performance is quantified using application efficiency, i.e., the percentage of maximum performance achieved, and the Minimum Effective Task Granularity (METG), i.e., the smallest task duration before runtime overheads dominate. Programmer productivity is quantified using Lines of Code (LOC) and the Number of Library Constructs (NLC). Our results reveal distinct trade-offs. MPI achieves the highest efficiency for regular, communication-light workloads but requires verbose, lowlevel code. HPX maintains stable efficiency under load imbalance across varying node counts, yet ranks last in productivity metrics, demonstrating that AMTs do not inherently guarantee improved productivity over MPI. Itoyori achieves the highest efficiency in communication-intensive configurations while leading in programmer productivity. ItoyoriFBC exhibits slightly lower efficiency than Itoyori, though its future-based synchronization offers potential for expressing irregular workloads.
Paper Structure (15 sections, 5 figures, 1 table)

This paper contains 15 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Implementation
  4. MPI
  5. HPX
  6. Itoyori
  7. ItoyoriFBC
  8. Experiments
  9. Varying Width and METG
  10. Weak Scaling
  11. Load Imbalance
  12. Communication-Intensive Benchmarks
  13. Discussion
  14. Related Work
  15. Conclusions

Figures (5)

  • Figure 1: Efficiency and METG for varying widths with a stencil graph, compute_bound kernel, and one node
  • Figure 2: Weak scaling from 1 to 16 nodes for a stencil graph, compute_bound kernel, and 16 width per core
  • Figure 3: Performance with load_imbalance kernels for a stencil graph, $2^{20}$ iterations, and 16 width per core
  • Figure 4: Communication-focused experiment with a spread graph, compute_bound kernel, $2^{20}$ iterations, 1 KiB output size, and 16 width per core
  • Figure 5: Communication-focused experiment with an all_to_all graph, compute_bound kernel, $2^{20}$ iterations, and 16 B output size