Benchmarking Quantum Computer Simulation Software Packages: State Vector Simulators

TL;DR

A containerized toolchain for benchmarking a large set of simulation packages on a local HPC cluster using different parallelisation capabilities is developed, and the performance and system size-scaling for three paradigmatic quantum computing tasks are compared.

Abstract

Rapid advances in quantum computing technology lead to an increasing need for software simulators that enable both algorithm design and the validation of results obtained from quantum hardware. This includes calculations that aim at probing regimes of quantum advantage, where a quantum computer outperforms a classical computer in the same task. High performance computing (HPC) platforms play a crucial role as today's quantum devices already reach beyond the limits of what powerful workstations can model, but a systematic evaluation of the individual performance of the many offered simulation packages is lacking so far. In this Technical Review, we benchmark several software packages capable of simulating quantum dynamics with a special focus on HPC capabilities. We develop a containerized toolchain for benchmarking a large set of simulation packages on a local HPC cluster using different parallelisation capabilities, and compare the performance and system size-scaling for three paradigmatic quantum computing tasks. Our results can help finding the right package for a given simulation task and lay the foundation for a systematic community effort to benchmark and validate upcoming versions of existing and also newly developed simulation packages.

Figures (10)

• Figure 1: Workflow of the toolchain which takes four inputs: (i) Quantum Task, (ii) Simulation package, (iii) Compute capability of the HPC, (iv) Float precision. The task is input as the QASM instruction set which is processed by the translator to further generate the package specific instruction set. Post translation, other auxiliary file generation is triggered that creates the necessary job scripts, linker files that scope the translated run files into the environment of the container. In addition, a script to trigger the job submission onto the cluster is also generated. Finally, the performance characteristics either corresponding to the time or memory are collected for further analysis. ($^{*}$) The translated files generated from OpenQASM are executed using the package installed on the container, while the pre- and post-processing steps are executed on the HPC node independent of the container.
• Figure 2: An operational example of the translator, translating the openQASM instructions into the instruction set of the package, $\textit{cirq}$. The translator incorporates timer functions at the start and the end of the circuit execution.
• Figure 3: Scaling of the number of gates for each task as a function of system size. The number of single qubit gates (SQG) and two qubit gates (TQG) are illustrated as a very simple proxy for the complexity of the calculations that have to be performed.
• Figure 4: Extraction of relative speed and scaling overhead. Performance time scales linearly in the large-$N$ limit on the $\log$ scale i.e., $\log (t) = a + bN$ where $t$ is the wall-clock time and $N$ is the system size (number of qubits). We extract the $y$-intercept, $a$ and slope, $b$ using the above linear fit, which we use for relative speed assessment (marked by the black solid line around $N=0$).
• Figure 5: (a) Singlethread performance (single precision). (top) Absolute wall clock time as a function of qubit number for packages that support single precision according to the package documentation. (bottom) Performance ratio with respect to best performing package, here $\textit{qsimcirq}$. (b) Scaling behavior of singlethread single precision performance by extracting $a$, $b$ as detailed in the main text. (top) Relative slow-down with respect to the fastest large-$N$ package, $\textit{qsimcirq}$. (bottom) Deviation from the expected $\log(2)$ exponential scaling (red line) as a function of the number of qubits.