Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Empirical Study: MEMS as a Static Performance Metric

Liwei Zhang, Baoquan Cui, Xutong Ma, Jian Zhang

TL;DR

The paper tackles architecture-dependent runtime variability by proposing mems, a machine-independent metric that counts memory accesses to estimate performance at compile time. It develops a Clang-based instrumentation pipeline (eppather-clangpass) to collect path-length, mems, and timing data, and uses symbolic execution with model counting to compute path frequencies and an expected performance via $Performance = \frac{\sum_i (\delta_i \cdot \text{pind}_i)}{\sum_i \delta_i}$. The study validates mems on ten classical C benchmarks, finding strong mems–time signals within individual programs but weaker generalization across different programs, influenced by data locality, input size, and hardware effects. The work proposes mems as a lightweight early-stage performance indicator, while acknowledging the need to combine it with additional metrics for robust, cross-program reasoning and WCET-like analyses.

Abstract

Static performance estimation is essential during compile-time analysis, yet traditional runtime-based methods are costly and platform-dependent. We investigate mems, the number of memory accesses, as a static and architecture-independent performance metric. We develop a Clang-based automated instrumentation tool that rewrites source code to insert path tracing and \textit{mems} counting logic. This allows us to evaluate mems-based performance estimation across ten classical algorithm programs. Experimental results show that within the same program, execution paths with higher mems values consistently exhibit longer runtime. However, this correlation weakens between different programs, suggesting that mems is best suited for comparing performance of different execution paths in a program.

An Empirical Study: MEMS as a Static Performance Metric

TL;DR

The paper tackles architecture-dependent runtime variability by proposing mems, a machine-independent metric that counts memory accesses to estimate performance at compile time. It develops a Clang-based instrumentation pipeline (eppather-clangpass) to collect path-length, mems, and timing data, and uses symbolic execution with model counting to compute path frequencies and an expected performance via . The study validates mems on ten classical C benchmarks, finding strong mems–time signals within individual programs but weaker generalization across different programs, influenced by data locality, input size, and hardware effects. The work proposes mems as a lightweight early-stage performance indicator, while acknowledging the need to combine it with additional metrics for robust, cross-program reasoning and WCET-like analyses.

Abstract

Static performance estimation is essential during compile-time analysis, yet traditional runtime-based methods are costly and platform-dependent. We investigate mems, the number of memory accesses, as a static and architecture-independent performance metric. We develop a Clang-based automated instrumentation tool that rewrites source code to insert path tracing and \textit{mems} counting logic. This allows us to evaluate mems-based performance estimation across ten classical algorithm programs. Experimental results show that within the same program, execution paths with higher mems values consistently exhibit longer runtime. However, this correlation weakens between different programs, suggesting that mems is best suited for comparing performance of different execution paths in a program.
Paper Structure (25 sections, 2 equations, 6 figures, 6 tables)

This paper contains 25 sections, 2 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. The mems Metric
  4. Path-Based Performance Estimation
  5. Example
  6. Remarks
  7. Approach
  8. Instrumentation Pipeline
  9. Instrumentation Example
  10. Evaluation
  11. Experimental Setup
  12. Program Benchmarks
  13. Execution Environments
  14. Measurement Protocol
  15. RQ1: More Memory Accesses, Longer Execution Time
  16. ...and 10 more sections

Figures (6)

  • Figure 1: Graphical representation of memory access frequency and execution time correlation.
  • Figure 2: Analysis of path length impact on execution time across different test conditions.
  • Figure 3: Scatter plots illustrating the relationship between mems, path_length, and execution time across different experimental scenarios.
  • Figure 4: Scatter plots depicting the relationship between mems and execution time after grouping data by similar path_length.
  • Figure 5: Scatter Plot of Input Size vs. Execution Time
  • ...and 1 more figures