Comparative Study of Quantum Transpilers: Evaluating the Performance of qiskit-braket-provider, qBraid-SDK, and Pytket Extensions
Mohamed Messaoud Louamri, Nacer Eddine Belaloui, Abdellah Tounsi, Mohamed Taha Rouabah
TL;DR
The paper tackles the lack of systematic benchmarks for SDK-to-SDK quantum transpilers by introducing RandomQC and Benchmarq to enable reproducible, diverse evaluations. It benchmarks key tools—qiskit-braket-provider, qBraid-SDK, and pytket extensions—across multiple circuit types and SDK pairs, focusing on correctness, failure rate, and transpilation time. The study finds that the specialized qiskit-braket-provider delivers the lowest failure rate and strong speed, while the generalized qBraid-SDK offers broad compatibility with moderate speed, and pytket extensions trade faster transpilation for higher failure on complex circuits. These findings inform practical guidelines for users and developers, suggesting that a hybrid strategy (one-to-one transpilation with gate decomposition) can maximize capabilities and efficiency across SDK-to-SDK transpilation tasks.
Abstract
In this study, we present a comprehensive evaluation of popular SDK-to-SDK quantum transpilers (that is transpilers that takes a quantum circuit from an initial SDK and output a quantum circuit in another SDK), focusing on critical metrics such as correctness, failure rate, and transpilation time. To ensure unbiased evaluation and accommodate diverse quantum computing scenarios, we developed two dedicated tools: RandomQC, for generating random quantum circuits across various types (pure random, VQE-like, and SDK-specific circuits), and Benchmarq, to streamline the benchmarking process. Using these tools, we benchmarked prominent quantum transpilers as of February 2024. Our results highlight the superior performance of the qiskit-braket-provider, a specialized transpiler from Qiskit to Braket, achieving a remarkably low failure rate of 0.2%. The qBraid-SDK, offering generalized transpilation across multiple SDKs, demonstrated robust but slower performance. The pytket extensions, while fast, faced limitations with complex circuits due to their one-to-one transpilation approach. In particular, the exceptional performance of the qiskit-bracket-provider stems not only from its specialization but also from its architecture, which combines one-to-one transpilation with gate decomposition for unsupported gates, enhancing both speed and capability. This study aims to provide practical guidelines to users of SDK-to-SDK quantum transpilers and guidance to developers for improving the design and development of future tools.