Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TALP-Pages: An easy-to-integrate continuous performance monitoring framework

Valentin Seitz, Jordy Trilaksono, Marta Garcia-Gasulla

TL;DR

This paper introduces TALP-Pages, an easy-to-integrate framework that enables developers to get fast and in-repository feedback about their code performance using established fundamental performance and scaling factors and finds that TALP-Pages can produce the scaling-efficiency tables faster and under tighter resource constraints.

Abstract

Ensuring good performance is a key aspect in the development of codes that target HPC machines. As these codes are under active development, the necessity to detect performance degradation early in the development process becomes apparent. In addition, having meaningful insight into application scaling behavior tightly coupled to the development workflow is helpful. In this paper, we introduce TALP-Pages, an easy-to-integrate framework that enables developers to get fast and in-repository feedback about their code performance using established fundamental performance and scaling factors. The framework relies on TALP, which enables the on-the-fly collection of these metrics. Based on a folder structure suited for CI which contains the files generated by TALP, TALP-Pages generates an HTML report with visualizations of the performance factor regression as well as scaling-efficiency tables. We compare TALP-Pages to tracing-based tools in terms of overhead and post-processing requirements and find that TALP-Pages can produce the scaling-efficiency tables faster and under tighter resource constraints. To showcase the ease of use and effectiveness of this approach, we extend the current CI setup of GENE-X with only minimal changes required and showcase the ability to detect and explain a performance improvement.

TALP-Pages: An easy-to-integrate continuous performance monitoring framework

TL;DR

This paper introduces TALP-Pages, an easy-to-integrate framework that enables developers to get fast and in-repository feedback about their code performance using established fundamental performance and scaling factors and finds that TALP-Pages can produce the scaling-efficiency tables faster and under tighter resource constraints.

Abstract

Ensuring good performance is a key aspect in the development of codes that target HPC machines. As these codes are under active development, the necessity to detect performance degradation early in the development process becomes apparent. In addition, having meaningful insight into application scaling behavior tightly coupled to the development workflow is helpful. In this paper, we introduce TALP-Pages, an easy-to-integrate framework that enables developers to get fast and in-repository feedback about their code performance using established fundamental performance and scaling factors. The framework relies on TALP, which enables the on-the-fly collection of these metrics. Based on a folder structure suited for CI which contains the files generated by TALP, TALP-Pages generates an HTML report with visualizations of the performance factor regression as well as scaling-efficiency tables. We compare TALP-Pages to tracing-based tools in terms of overhead and post-processing requirements and find that TALP-Pages can produce the scaling-efficiency tables faster and under tighter resource constraints. To showcase the ease of use and effectiveness of this approach, we extend the current CI setup of GENE-X with only minimal changes required and showcase the ability to detect and explain a performance improvement.
Paper Structure (24 sections, 7 figures, 5 tables)

This paper contains 24 sections, 7 figures, 5 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Background
  4. TALP-Pages
  5. Comparison to other tools
  6. Integration into GENE-X
  7. Discussion
  8. Conclusions and future work
  9. Acknowledgement
  10. Appendix

Figures (7)

  • Figure 1: Abstract workflow for a continuous performance monitoring framework.
  • Figure 2: Exemplary folder structure suitable as TALP-Pages input.
  • Figure 3: Scaling efficiency table generated by TALP-Pages of a MPI-only strong scaling experiment
  • Figure 4: Exemplary usage of TALP-Pages in a CI environment
  • Figure 5: Matrix job example of the the performance stage in the GENE-X pipeline
  • ...and 2 more figures