Pauli Network Circuit Synthesis with Reinforcement Learning

TL;DR

The paper tackles the challenge of efficiently transpiling quantum circuits by introducing a reinforcement-learning–based method to re-synthesize Pauli Networks that include Clifford gates and single-qubit rotations up to six qubits. It adapts a PPO-based RL framework to learn a gate-placing heuristic, uses a structured state representation combining a Clifford tableau and a commutation DAG, and incorporates a collect-and-re-synthesize pipeline to optimize large circuits under hardware coupling constraints. Empirically, the approach yields substantial improvements: on 6-qubit random Pauli Networks it achieves roughly a 2x reduction in two-qubit gate count relative to heuristics, and when embedded in a post-transpilation Benchpress workflow it provides average 20% reductions in two-qubit gate count and depth (with larger gains in many instances). These results demonstrate the practicality and scalability of RL-driven local circuit resynthesis for realistic, large-scale quantum transpilation workloads.

Abstract

We introduce a Reinforcement Learning (RL)-based method for re-synthesis of quantum circuits containing arbitrary Pauli rotations alongside Clifford operations. By collapsing each sub-block to a compact representation and then synthesizing it step-by-step through a learned heuristic, we obtain circuits that are both shorter and compliant with hardware connectivity constraints. We find that the method is fast enough and good enough to work as an optimization procedure: in direct comparisons on 6-qubit random Pauli Networks against state-of-the-art heuristic methods, our RL approach yields over 2x reduction in two-qubit gate count, while executing in under 10 milliseconds per circuit. We further integrate the method into a collect-and-re-synthesize pipeline, applied as a Qiskit transpiler pass, where we observe average improvements of 20% in two-qubit gate count and depth, reaching up to 60% for many instances, across the Benchpress benchmark. These results highlight the potential of RL-driven synthesis to significantly improve circuit quality in realistic, large-scale quantum transpilation workloads.

Figures (14)

• Figure 1:
• Figure 2:
• Figure 3: Example of agent update. The agent starts by adding a CNOT to the output circuit. The tableau is updated accordingly. We then notice that the first rotation (which lies in the front layer of the DAG) became trivial (i.e. a single qubit rotation). The corresponding rotation is added to the output circuit, the Pauli is cleared, and $1$ is removed from the DAG. The second rotation is now part of the front layer and is also trivial, leading to the same treatment, finalizing the update. The agent inserts a last CNOT, the tableau is updated. The third rotation is now part of the front layer and trivial, leading to the injection of a last $R_Y$ into the output circuit. After those two moves, the Clifford tableau (left tableau) is trivial (i.e. implements the identity) and the DAG is empty: the synthesis process is over.
• Figure 4: Pauli Network collection pass on an arbitrary circuit. The collected sub-circuits are to be re-synthesized by the RL agent to optimize on count or depth.
• Figure 5: Number of two-qubit gates per Pauli rotation for the different algorithms and coupling maps, for full IXYZ Paulis.