SynPerf: A Hybrid Analytical-ML Framework for GPU Performance Prediction

TL;DR

SynPerf addresses the need for fast, accurate GPU performance modeling in large-scale LLM serving by fusing an analytical kernel decomposition with a lightweight ML predictor. It decomposes kernels into tasks, simulates scheduling, derives per-pipeline demands for Math and MIO pipelines, and uses a compact MLP to predict latency. Across 11 GPUs and multiple kernel types, SynPerf achieves kernel-level MAPEs around $6\%$ on seen hardware and around $11\%$ on unseen hardware, outperforming the prior state-of-the-art by up to about $6\times$ for kernel-level predictions and $4\times$ for end-to-end predictions. It also defines a potential performance ceiling via quantile regression (P80) to diagnose and guide optimizations, demonstrated by a production speedup of up to $1.7\times$ on a fused MoE kernel. The framework enables fast hardware screening and practical system-level exploration for next-generation GPUs in large-scale LLM serving.

Abstract

The rapid expansion of Transformer-based large language models has dramatically increased the need for high-performance GPUs. As a result, there is growing demand for fast, accurate, and widely generalizable GPU performance models to support next-generation hardware selection and system-level exploration. However, current data-driven methods are limited, exhibiting poor generalization across hardware and inadequate modeling of complex production-level kernels common in modern inference stacks. To address these issues, we present SyncPerf, a unified GPU modeling framework. This approach first employs an analytical model to quantify a given kernel's demands on the GPU's heterogeneous instruction pipelines. These analytical features are then fed into a machine learning (ML) model to capture complex cross-pipeline interactions and resource dependencies, enabling high-fidelity performance prediction. Our evaluation across 11 GPU types from four generations of major architectures on two widely-used serving systems demonstrates that SyncPerf delivers high fidelity and strong generalizability. It achieves accurate predictions, with only 6.1% average error at the kernel level and 8.5% for end-to-end inference -- reducing the error of state-of-the-art methods by 6.7x and 4.4x, respectively. We also demonstrate SynPerf's value "beyond simulation" by utilizing its performance ceiling to diagnose implementation shortcomings and guide the optimization of a production fused MoE Triton kernel, achieving up to 1.7x speedup.

Figures (8)

• Figure 1: An illustration of the mapping between the software hierarchy and the physical GPU hardware hierarchy.
• Figure 2: Overview of the SynPerf modeling framework, detailing the flow from kernel decomposition to the final performance prediction.
• Figure 3: Illustration of the SynPerf multi-dimensional analysis for FlashAttention-2 on A100. As demand increases, measured performance for two different configurations approaches the theoretical "roof" and plateaus.
• Figure 4: Ablation study on the impact of MIO and Math Pipeline features for GEMM and Attention kernels.
• Figure 5: Kernel-level prediction accuracy (MAPE) of SynPerf and baseline models. Unseen hardware platforms are identified by a grey background.