Hybrid Quantum-HPC Middleware Systems for Adaptive Resource, Workload and Task Management

Abstract

Hybrid quantum-classical applications pose significant resource management challenges due to heterogeneity and dynamism in both infrastructure and workloads. Quantum-HPC environments integrate quantum processing units (QPUs) with diverse classical resources (CPUs, GPUs), while applications span coupling patterns from tightly coupled execution to loosely coupled task parallelism with varying resource requirements. Traditional HPC schedulers lack visibility into application semantics and cannot respond to fluctuating resource availability at runtime. This paper presents a middleware-based approach for adaptive resource, workload, and task management in hybrid quantum-HPC systems. We make four contributions: (i) a conceptual four-layer middleware architecture that decomposes management across workflow, workload, task, and resource levels, enabling application-aware scheduling over heterogeneous quantum-HPC resources; (ii) a set of execution motifs capturing interaction and coupling characteristics of hybrid applications, realized as quantum mini-apps for systematic workload characterization; (iii) Pilot-Quantum, a middleware framework built on the pilot abstraction that enables late binding and dynamic resource allocation, adapting to resource and workload dynamics at runtime; and (iv) Q-Dreamer, a performance modeling toolkit providing reusable components for informed workload partitioning, including a circuit-cutting optimizer that analytically derives optimal partitioning strategies. Evaluation on heterogeneous HPC platforms (Perlmutter, NVIDIA DGX with H100/B200 GPUs) demonstrates efficient multi-backend orchestration across CPUs, GPUs, and QPUs for diverse execution motifs. Q-Dreamer predicts optimal circuit cutting configurations with up to 82% accuracy.

Figures (11)

• Figure 1: Quantum-HPC Integration Patterns: HPC-for-Quantum requires interactions within the coherence time of the QPU, Quantum-in-HPC a mix of classical and quantum tasks that need to be orchestrated, Quantum-about-HPC connects composable tasks to workflows.
• Figure 2: Quantum Software Stack: An overview of key components and layers, from quantum programming environments and hybrid runtimes to hardware resource management, including emerging standards.
• Figure 3: Basic and Compositional Execution Motifs for Quantum-HPC Workflows: Basic motifs represent fundamental patterns of quantum computation and classical-quantum interaction, including circuit execution, distributed simulation, circuit cutting, and error mitigation. These motifs are characterized by their coupling intensity (tight vs. loose) and interaction patterns (concurrent vs. sequential). Compositional motifs orchestrate multiple basic motifs to create real-world applications, such as multi-stage pipelines, parallel VQAs with synchronous or asynchronous coordination, and generative quantum algorithms that combine classical machine learning with quantum circuit execution.
• Figure 4: Pilot-Quantum Architecture: The system core is a Pilot-Manager that orchestrates and manages resources through pilots across both classical and quantum infrastructures, such as QPUs, GPUs, and CPUs. Pilots are responsible for reserving resources and managing task execution.
• Figure 5: Q-Dreamer Toolkit Architecture: The Q-Dreamer framework consists of two main layers. The Q-Dreamer Core layer provides re-usable building blocks for resource detection and workload analysis. The Workload Management Tools layer provides application- and workload-specific tools to support scheduling decisions based on the Q-Dreamer core.