Quantum Compiling with Reinforcement Learning on a Superconducting Processor

TL;DR

The paper tackles efficient quantum circuit compilation for NISQ devices by introducing a reinforcement-learning-based compiler that combines a deep Q-network with AQ* search tailored to a superconducting processor's native gate set and connectivity. It demonstrates hardware-aware, short-depth circuit synthesis, achieving notable results such as a three-qubit QFT using seven CZ gates with unity fidelity and superior performance under topological constraints compared with conventional methods. The approach includes a variational post-processing extension (VRL) that further improves fidelity by optimizing single-qubit gate parameters. Overall, the work exemplifies software-hardware co-design for quantum compilation on real devices and offers a scalable framework for RL-based compilers on multiqubit systems.

Abstract

To effectively implement quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we develop a reinforcement learning (RL)-based quantum compiler for a superconducting processor and demonstrate its capability of discovering novel and hardware-amenable circuits with short lengths. We show that for the three-qubit quantum Fourier transformation, a compiled circuit using only seven CZ gates with unity circuit fidelity can be achieved. The compiler is also able to find optimal circuits under device topological constraints, with lengths considerably shorter than those by the conventional method. Our study exemplifies the codesign of the software with hardware for efficient quantum compilation, offering valuable insights for the advancement of RL-based compilers.

Figures (17)

• Figure 1: Architecture of the RL-based quantum compiler. (a) Three metrics to measure the quality of a quantum compiler. (b) Precompilation. The action space is first initialized based on the native gate set and circuit topology of the superconducting processor. Then the agent starts the training of deep Q-network (DQN). Namely, DQN constantly queries the database, highlighted by two boxes 'Percept' and 'Value' in which small cuboids with different colors and heights represent different unitaries and values, to generate training data by pairing unitaries with the information of correct gate decompositions. The data training is from easy to hard along the downward arrow. (c) Inference. Given an unknown target unitary, the RL agent employs the AQ* search guided by the trained DQN to predict the decomposed gate sequence, which is finally implemented in the quantum processor. Here, compilation for the two-qubit system is illustrated for simplicity.
• Figure 2: Compilation for two-qubit system. (a) SWAP gate. The blue-colored circuit represents the standard decomposition, while the pink-colored circuit shows the result by RL compilation with the inference time of $\sim 1.23$ seconds. The lower-left panel displays the statistical results of fidelity $F_2$ measured on Q$_1$-Q$_2$ for each circuit with the same color, and histograms drawn with dashed lines refer to the results of $F_1$. (b) Similar results for $\text{R}_{\text{ZZ}}(\pi/{2})$ gate with pink and green colors representing two gate sequences returned by the RL compiler. The inference times are comparable to that for the SWAP gate.
• Figure 3: Compilation for the three-qubit system. (a) QFT circuits by Qiskit and RL compiler (inference time $\sim$ 800 seconds). Squares denote single-qubit gates from $\mathcal{G}$. (b) Performance analysis of the QFT circuits on Q$_2$-Q$_3$-Q$_6$ qubits using the total variance (TV) metric. The QFT circuit is evaluated on four varied input states $|000\rangle$, $|$+$00\rangle$, $|$++$0\rangle$, $|$+++$\rangle$ and $\hat{U}'|000\rangle$, where $\hat{U}'$ is randomly sampled from SU(8). Here, $d_{\text{TV}}(P_i, P_u)$ quantifies the TV distance between the distributions of the ideal output state $P_i$ and the uniform distribution $P_u$ in the measurement of computational basis. For each panel in Subplot (b), the notations $d_{\text{TV,Qiskit}}$, $d_{\text{TV,RL}}$, and $d_{\text{TV,VRL}}$ represent $d_{\text{TV}}(P_e, P_u)$ in which $P_e$ refers to the distribution of the generated state for QFT circuits compiled by Qiskit, RL, and VRL compilers, respectively. A lower difference with $d_{\text{TV}}(P_i, P_u)$ suggests a better performance. (c) Statistical performance comparison of Qiskit, RL, and VRL by preparing arbitrary input states. The notaions in x-axis and y-axis follow those in Subplot (b). The inset shows the fidelity (dashed line refers to $F_1$ and histogram refers to $F_2$) of the QFT circuits compiled by three indicated compilers. Subplots (d) and (e) analyze the statistical performance of Qiskit and RL compilers for compiling $10$ two-qubit gates with topological constraints under the metric of $F_1$ (dashed line), $F_2$ and $N_2$.
• Figure 4: Error analysis. (a) Experimental fidelity measured by QPT versus the unit circuit depth with random circuits shown in (b) for Q$_2$-Q$_3$-Q$_6$. At the zeroth depth, only two $\text{R}_{\phi}(\pi/2)$ gates are applied to each qubit, where $\text{R}_{\phi}(\pi/2)=\exp({-i\frac{\pi}{4}(\cos{\phi}\text{X}+\sin{\phi}\text{Y})})$ and $\phi$ refers to a set of specific angles. The star indicates the circuit in Fig. \ref{['fig:three_qubit_experiment']}(a) by RL compiler. (c) Experimental fidelity of QFT circuit measured by QPT versus averaged CZ gate fidelity for seven three-qubit sets of Q$_2$-Q$_1$-Q$_4$, Q$_2$-Q$_3$-Q$_6$, Q$_3$-Q$_6$-Q$_5$, Q$_4$-Q$_5$-Q$_2$, Q$_4$-Q$_5$-Q$_6$, Q$_5$-Q$_2$-Q$_3$, and Q$_6$-Q$_5$-Q$_2$. The line is a linear fit. (d) Circuit depth dependences of fidelity measured using XEB for two three-qubit sets. The lines fit as discussed in the text.
• Figure S1: Experimental setup. Electronics and wirings for synthesizing and transmitting the control/readout signals. Each qubit has three control channels: XY (microwave pulse), fast Z (Z pulse), and slow Z (direct current). Each coupler has two control channels: fast Z and slow Z. Readout pulses are generated in a similar way to the XY signals and are passed through the processor via the readout transmission lines. All the control and readout lines are well attenuated and filtered for noise shielding and precise control. The superconducting processor shown at the bottom has 9 qubits (crosses) and 12 couplers (rectangles) arranged in a 3$\times$3 lattice configuration. The qubit index is indicated and the coupler between Q$_i$ and Q$_j$ will be denoted as C$_{ij}$.