Pilot-Quantum: A Quantum-HPC Middleware for Resource, Workload and Task Management

TL;DR

Pilot-Quantum introduces a modular quantum-HPC middleware built on the Pilot Abstraction to unify resource, workload, and task management across heterogeneous classical and quantum resources. By separating concerns across four architectural layers and supporting high-level frameworks (Qiskit, PennyLane) as well as low-level runtimes (MPI, CUDA, cuQuantum), it enables scalable, adaptive orchestration of hybrid quantum-classical workflows. The authors validate the approach with mini-apps that span circuit execution, circuit cutting, distributed state-vector simulation, and QML, reporting meaningful performance improvements and flexible resource utilization. The work highlights the importance of multi-level scheduling and application-level resource management to address hardware and software heterogeneity in evolving quantum-HPC environments. Future directions include exposing QPU cloud resources through standard quantum languages and integrating higher-level workflow frameworks to further enhance scheduling and interoperability.

Abstract

As quantum hardware advances, integrating quantum processing units (QPUs) into HPC environments and managing diverse infrastructure and software stacks becomes increasingly essential. Pilot-Quantum addresses these challenges as a middleware designed to provide unified application-level management of resources and workloads across hybrid quantum-classical environments. It is built on a rigorous analysis of existing quantum middleware systems and application execution patterns. It implements the Pilot Abstraction conceptual model, originally developed for HPC, to manage resources, workloads, and tasks. It is designed for quantum applications that rely on task parallelism, including (i) hybrid algorithms, such as variational approaches, and (ii) circuit cutting systems, used to partition and execute large quantum circuits. Pilot-Quantum facilitates seamless integration of QPUs, classical CPUs, and GPUs, while supporting high-level programming frameworks like Qiskit and Pennylane. This enables users to efficiently design and execute hybrid workflows across diverse computing resources. The capabilities of Pilot-Quantum are demonstrated through mini-apps -- simplified yet representative kernels focusing on critical performance bottlenecks. We demonstrate the capabilities of Pilot-Quantum through multiple mini-apps, including different circuit executions (e.g., using IBMś Eagle QPU and simulators), circuit cutting, and quantum machine learning scenarios.

Figures (8)

• Figure 1: Quantum Software Stack: An overview of key components and layers, from quantum programming environments and hybrid runtimes to hardware resource management.
• Figure 2: Pilot Quantum Architecture: The system core is a Pilot-Manager that orchestrates and manages resources through pilots across both classical and quantum infrastructures, such as QPUs, GPUs, and CPUs. Pilots are responsible for reserving resources and managing task execution.
• Figure 3: Circuit Execution on IonQ, IBM Eagle and Qiskit Aer (CPU and GPU): Comparing the execution times for 1,024 random quantum circuits (2 to 28 qubits) on IonQ Quantum Cloud and Qiskit's Aer simulators (CPU and GPU) using Pilot-Quantum (Ray) on one Perlmutter node. We only executed eight random circuits for each qubit configuration for IBM Eagle and scaled their mean execution time to be comparable with the 1,024-circuit experiments. IonQ required 4,260 seconds for 28 qubits, while Aer (CPU) took 470 seconds, and Aer (GPU) completed it in just 49 seconds. The scaled execution time of the IBM Eagle QPU is 199 seconds.
• Figure 4: PennyLane lightning.gpu Distributed State Vector Simulations: Computing the expectation value of a 2-layer strongly entangling layered (SEL) circuit with and without gradient calculation using PennyLane's lightning.gpu device on 256 A100 40 GB GPUs on Perlmutter.
• Figure 5: Qiskit Circuit Cutting technique with Pilot-Quantum: Illustrates performance of circuit cutting for a 34 qubit circuit using 2 cuts/72 subexperiments, 4 cuts/2100 subexperiments, and full circuit simulation using Pilot-Quantum/Ray on Perlmutter 4x A100 80 GB GPUs nodes and Qiskit's AerSimulator (with CUDA/MPI support). For circuit cutting, we use one GPU per subexperiment. For full circuit simulation, we use 4 GPUs/node and distributed state vector simulation.