Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthesizing Proxy Applications for MPI Programs

Jiyu Luo, Tao Yan, Qingguo Xu, Jingwei Sun, Guangzhong Sun

TL;DR

This work addresses the challenge of evaluating HPC systems with proxy-apps by automating proxy-app synthesis from MPI traces. It introduces Siesta, which traces MPI communication via PMPI, abstracts computation with MPI_Compute footprints derived from PAPI counters, and composes proxies by solving a convex optimization over a predefined set of code blocks. Siesta compresses intra-process patterns with a Sequitur-based grammar and merges inter-process patterns to produce a compact, executable proxy-app that faithfully mirrors both communication and computation characteristics (average relative error around 2.48%). The approach outperforms existing proxy-app synthesis and trace-compression methods in fidelity and robustness across MPI implementations, enabling more accurate performance modeling and system design exploration for HPC workloads.

Abstract

Proxy applications (proxy-apps) are basic tools for evaluating the performance of specific workloads on high-performance computing (HPC) systems. Since the development of high-fidelity proxy-apps, which exhibit similar performance characteristics as corresponding production applications, is labor-intensive, synthetic proxy-apps are created as a useful supplement to manually developed proxy-apps. To thoroughly resemble performance characteristics of HPC applications represented by Message Passing Interface (MPI) programs, we propose Siesta, a novel framework to automatically synthesize proxy-apps based on communication-computation traces. Given an MPI program, Siesta synthesizes parameterized code snippets to mimic computation behaviors in different execution periods, and combines the code snippets and MPI function records into an event trace. It then extracts program behavior patterns from the trace as grammars and finally transforms the grammars into a synthetic proxy-app. We evaluate the proposed methods on representative MPI programs with various environments. The results show that our synthetic proxy-apps can precisely approximate the performance characteristics of MPI programs.

Synthesizing Proxy Applications for MPI Programs

TL;DR

This work addresses the challenge of evaluating HPC systems with proxy-apps by automating proxy-app synthesis from MPI traces. It introduces Siesta, which traces MPI communication via PMPI, abstracts computation with MPI_Compute footprints derived from PAPI counters, and composes proxies by solving a convex optimization over a predefined set of code blocks. Siesta compresses intra-process patterns with a Sequitur-based grammar and merges inter-process patterns to produce a compact, executable proxy-app that faithfully mirrors both communication and computation characteristics (average relative error around 2.48%). The approach outperforms existing proxy-app synthesis and trace-compression methods in fidelity and robustness across MPI implementations, enabling more accurate performance modeling and system design exploration for HPC workloads.

Abstract

Proxy applications (proxy-apps) are basic tools for evaluating the performance of specific workloads on high-performance computing (HPC) systems. Since the development of high-fidelity proxy-apps, which exhibit similar performance characteristics as corresponding production applications, is labor-intensive, synthetic proxy-apps are created as a useful supplement to manually developed proxy-apps. To thoroughly resemble performance characteristics of HPC applications represented by Message Passing Interface (MPI) programs, we propose Siesta, a novel framework to automatically synthesize proxy-apps based on communication-computation traces. Given an MPI program, Siesta synthesizes parameterized code snippets to mimic computation behaviors in different execution periods, and combines the code snippets and MPI function records into an event trace. It then extracts program behavior patterns from the trace as grammars and finally transforms the grammars into a synthetic proxy-app. We evaluate the proposed methods on representative MPI programs with various environments. The results show that our synthetic proxy-apps can precisely approximate the performance characteristics of MPI programs.
Paper Structure (24 sections, 5 equations, 9 figures, 3 tables)

This paper contains 24 sections, 5 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method Implementation
  3. Overview
  4. Tracing Communication Events
  5. Tracing Computation Events
  6. Searching Computation Proxies
  7. Extracting Intra-process Behavior Patterns
  8. Extracting Inter-process Behavior Patterns
  9. Merging Terminal Tables
  10. Merging Non-terminal Tables
  11. Code Generation
  12. Evaluation
  13. Setup
  14. Specification of Generated Proxy-apps
  15. Comparison of compression ability
  16. ...and 9 more sections

Figures (9)

  • Figure 1: Overview of proxy-app synthesis for MPI programs
  • Figure 2: Code block examples for mimicing behaviour of application
  • Figure 3: Example of merging two grammars
  • Figure 4: Code generation example with grammar in Equation \ref{['code_gen:grammar']}
  • Figure 5: Comparison of compressed data size between our method and other methods
  • ...and 4 more figures