Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BenchQC -- Scalable and modular benchmarking of industrial quantum computing applications

Florian Geissler, Eric Stopfer, Christian Ufrecht, Nico Meyer, Daniel D. Scherer, Friedrich Wagner, Johannes M. Oberreuter, Zao Chen, Alessandro Farace, Daria Gutina, Ulrich Schwenk, Kimberly Lange, Vanessa Junk, Thomas Husslein, Marvin Erdmann, Florian Kiwit, Benjamin Decker, Greshma Shaji, Etienne Granet, Henrik Dreyer, Theodora-Augustina Dragan, Jeanette Miriam Lorenz

TL;DR

BenchQC addresses the need for industry-relevant benchmarking of quantum computing by centering on real-world use cases and a full-stack workflow built on the QUARK platform. It introduces an application-centric methodology that ties problem formulations and problem-size scaling to hardware execution and application-level metrics, enabling systematic assessment of quantum utility. The paper details six use cases across optimization, machine learning, and simulation, reporting results that show both competitive performance in certain scenarios and actionable guidance for circuit- and hardware-design in others. By establishing benchmarking standards and providing modular tooling, BenchQC aims to accelerate the transition from demonstrations to practical quantum advantage in industrial contexts.

Abstract

We present BenchQC, a research project funded by the state of Bavaria, which promotes an application-centric perspective for benchmarking real-world quantum applications. Diverse use cases from industry consortium members are the starting point of a benchmarking workflow, that builds on the open-source platform QUARK, encompassing the full quantum software stack from the hardware provider interface to the application layer. By identifying and evaluating key metrics across the entire pipeline, we aim to uncover meaningful trends, provide systematic guidance on quantum utility, and distinguish promising research directions from less viable approaches. Ultimately, this initiative contributes to the broader effort of establishing reliable benchmarking standards that drive the transition from experimental demonstrations to practical quantum advantage.

BenchQC -- Scalable and modular benchmarking of industrial quantum computing applications

TL;DR

BenchQC addresses the need for industry-relevant benchmarking of quantum computing by centering on real-world use cases and a full-stack workflow built on the QUARK platform. It introduces an application-centric methodology that ties problem formulations and problem-size scaling to hardware execution and application-level metrics, enabling systematic assessment of quantum utility. The paper details six use cases across optimization, machine learning, and simulation, reporting results that show both competitive performance in certain scenarios and actionable guidance for circuit- and hardware-design in others. By establishing benchmarking standards and providing modular tooling, BenchQC aims to accelerate the transition from demonstrations to practical quantum advantage in industrial contexts.

Abstract

We present BenchQC, a research project funded by the state of Bavaria, which promotes an application-centric perspective for benchmarking real-world quantum applications. Diverse use cases from industry consortium members are the starting point of a benchmarking workflow, that builds on the open-source platform QUARK, encompassing the full quantum software stack from the hardware provider interface to the application layer. By identifying and evaluating key metrics across the entire pipeline, we aim to uncover meaningful trends, provide systematic guidance on quantum utility, and distinguish promising research directions from less viable approaches. Ultimately, this initiative contributes to the broader effort of establishing reliable benchmarking standards that drive the transition from experimental demonstrations to practical quantum advantage.

Paper Structure

This paper contains 24 sections, 7 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Consortium
  4. Related work
  5. Methodology
  6. Problem decomposition
  7. Metrics
  8. The QUARK platform
  9. Use cases: Optimization
  10. Basics of optimization
  11. Benchmarking Quantum Computing Optimization in the NISQ Era
  12. Metrics for Benchmarking Quantum Optimization
  13. Use Case: Portfolio Optimization
  14. Use Case: Assembly Line Balancing
  15. Use cases: Machine learning
  16. ...and 9 more sections

Figures (8)

  • Figure 1: Decomposition of the benchmarking workflow.
  • Figure 2: Metrics across the benchmarking pipeline. Asterisks (*) indicate that a form of this metric is currently implemented in QUARK.
  • Figure 3: Like every QUARK module, SCPQUBO inherits from Core and implements the preprocess and postprocess functions to interface with other modules in the pipeline. By also implementing the optional Serializable and Visualizable protocols, SCPQUBO clearly communicates its extra capabilities to QUARK.
  • Figure 4: The QUARK benchmarking pipeline is made up of modules that can pre- and postprocess data. In this case, the set cover problem is solved in two phases: By calling the preprocess functions of each module in order, a graph is created (1), mapped to a QUBO formulation (2), and solved on a simulated annealer (3). Afterwards, calling the postprocess functions of each module in reverse order, the lowest energy sample is obtained (4), re-interpreted as a list of nodes (5), and compared with the graph (6).
  • Figure 5: Runtime comparison for solving the Minvola, Maxret and Multiobj problem formulations of the portfolio optimization problem with SCIP bestuzheva_enabling_2023 with relative optimality gaps of $0\%$ and $5\%$.
  • ...and 3 more figures