Interactive and Urgent HPC: Challenges and Opportunities
Albert Reuther, Nick Brown, William Arndt, Johannes Blaschke, Christian Boehme, Antony Chazapis, Bjoern Enders, Robert Henschel, Julian Kunkel, Maxime Martinasso
TL;DR
The paper addresses the need to broaden HPC to time-sensitive, interactive, and urgent workloads. It surveys current practice across policy, scheduling, tooling, data management, and user support, and identifies gaps. It proposes research directions including new metrics for time-sensitive workflows, co-existing scheduling paradigms, standardized data interfaces, and enhanced user education. The work emphasizes building an open community and practical pilots to translate insights into production HPC systems.
Abstract
As a broader set of applications from simulations to data analysis and machine learning require more parallel computational capability, the demand for interactive and urgent high performance computing (HPC) continues to increase. This paper overviews the progress made so far and elucidates the challenges and opportunities for greater integration of interactive and urgent HPC policies, techniques, and technologies into HPC ecosystems.