Attention-Informed Surrogates for Navigating Power-Performance Trade-offs in HPC

TL;DR

Addresses HPC scheduling by balancing job runtime and power using a surrogate-driven MOBO framework. Employs attention-based embeddings via TabNet and intelligent sample acquisition to learn robust surrogates from noisy telemetry, improving hypervolume $HV$ of Pareto fronts. Demonstrates on two production HPC traces PM100 and Adastra that the method achieves higher $HV$ Pareto fronts and reduces training costs compared with baselines. Provides a data-efficient, end-to-end pipeline ready for integration into next-generation HPC schedulers to enable multi-objective optimization in production workloads.

Abstract

High-Performance Computing (HPC) schedulers must balance user performance with facility-wide resource constraints. The task boils down to selecting the optimal number of nodes for a given job. We present a surrogate-assisted multi-objective Bayesian optimization (MOBO) framework to automate this complex decision. Our core hypothesis is that surrogate models informed by attention-based embeddings of job telemetry can capture performance dynamics more effectively than standard regression techniques. We pair this with an intelligent sample acquisition strategy to ensure the approach is data-efficient. On two production HPC datasets, our embedding-informed method consistently identified higher-quality Pareto fronts of runtime-power trade-offs compared to baselines. Furthermore, our intelligent data sampling strategy drastically reduced training costs while improving the stability of the results. To our knowledge, this is the first work to successfully apply embedding-informed surrogates in a MOBO framework to the HPC scheduling problem, jointly optimizing for performance and power on production workloads.

Figures (6)

• Figure 1: End-to-end pipeline for surrogate-driven MOBO on HPC scheduling.
• Figure 2: Runtime surrogate convergence across different models.
• Figure 3: Power surrogate convergence across different models.
• Figure 4: Embedding-informed surrogates yield higher HV than TabNet models on PM100 and Adastra.
• Figure 5: MOBO achieves higher HV than SOBO on PM100, revealing better runtime–power trade-offs.