Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networks

TL;DR

The paper tackles quantum circuit optimization under noise by combining ZX-calculus, graph neural networks, and reinforcement learning to learn rule sequences that minimize $CNOT$ gates. It formulates ZX graph rewriting as an RL problem on ZX diagrams, using a tree search and PPO-based training to predict rule applications and positions, with extraction integrated via graph-like ZX diagrams. The approach demonstrates competitive improvements over state-of-the-art, generalizes to unseen gate ratios, and scales to larger circuits through peephole optimization, albeit with slower runtime. This work highlights a path toward discovering optimization rules beyond hand-crafted templates, enabling scalable, diagrammatic circuit optimization suitable for NISQ devices.

Abstract

Quantum computing is currently strongly limited by the impact of noise, in particular introduced by the application of two-qubit gates. For this reason, reducing the number of two-qubit gates is of paramount importance on noisy intermediate-scale quantum hardware. To advance towards more reliable quantum computing, we introduce a framework based on ZX calculus, graph-neural networks and reinforcement learning for quantum circuit optimization. By combining reinforcement learning and tree search, our method addresses the challenge of selecting optimal sequences of ZX calculus rewrite rules. Instead of relying on existing heuristic rules for minimizing circuits, our method trains a novel reinforcement learning policy that directly operates on ZX-graphs, therefore allowing us to search through the space of all possible circuit transformations to find a circuit significantly minimizing the number of CNOT gates. This way we can scale beyond hard-coded rules towards discovering arbitrary optimization rules. We demonstrate our method's competetiveness with state-of-the-art circuit optimizers and generalization capabilities on large sets of diverse random circuits.

Figures (6)

• Figure 1: Overview of our method. Our method iteratively constructs a tree in which every node corresponds to a different transformation of the original circuit. In every iteration, the agent first selects one of the transformed circuits (step 1) for further analysis. The chosen circuit gets processed by a GNN (step 2) which predicts both a ZX-calculus rule, and the position in the graph where the rule should be applied. After applying that rule (step 3) we obtain a transformed circuit. The new transformation is added as a child to the selected node (step 4). This loop is repeated $K$ times, after which the best circuit (wrt a quality function) is selected from the tree.
• Figure 2: The figure shows the rules available to the reinforcement-learning agent in this work. The arrows show in which direction the rules can be applied. ($f$) Spider fusion: Two spiders with the same color that are connected by at least one wire can be fused and the phases are added. ($uf$) Spider un-fusion: This transformation allows to partially reverse the spider-fusion rule. For a node with $n$ in or outgoing wires with $n>3$, the rule replaces the node with a complete graph with $n$ nodes, one of them holding the initial phase. By application of a sequence of spider-fusion steps, the agent can create any layout of two nodes whose fusion would result in the left-hand-side, at least in the case of zero phase. Note that our implementation currently does not support splitting of the phase. ($\pi$) $\pi$-commute: A spider with phase $\pi$ can be pushed through a spider of different color together with a change of the sign of the spider's phase. ($c$) color change: The color of a node can be changed by pushing a Hadamard gate on each in and outgoing wire. ($b$) bialgebra: This rule can be used to interchange X and Z spiders at the cost of adding many new nodes and edges. While this rules is very powerful, it typically strongly modifies the structure of the graph, making circuit extraction challenging. ($e$) Euler rule: A Hadamard gate can be decomposed into a sequence of spiders with phase $\pi/2$ of alternating color. All rules are implemented also for X and Z spiders swapped. In addition to those rules, we remove identity spiders (spiders with exactly two in or outgoing wires and zero phase) after each transformation step. Finally, we automatically remove any pair of wires connecting two spiders of different color. This rule can be derived from the copy rule Wetering2020 which would have to be added together with a generalized un-fusion rule to achieve completeness of the Clifford fragment of ZX calculus. Completeness in the general sense additionally requires a modification of the Euler rule Vilmart2019.
• Figure 3: The node selection process. First, the model predicts a weight $W_i$ for every node in isolation (a). We then accumulate the independent weights into path-weights from the root to each node with selectable actions (b). We then sample the state to explore from these accumulated weights by applying a softmax operation over weights (c). This is mathematically equivalent to imposing a random-walk from the root to a node under a probability distribution dependent on the $W_i$. After a node $n$ is sampled it gets expanded according to the rule and position produced by policy $\pi_{\text{zx}}(\text{rule}|n)$
• Figure 4: Histogram of two-qubit gates in circuits optimized by different methods. All circuits are from the dataset with a gate ratio of 1.0/0.0/0.0/0.0 consisting of 80 CNOT gates. The bins here show the number of two-qubit gates after optimization.
• Figure 5: Results on fully random circuits. Both our method and Staudacher2023 were evaluated using peepholes, while the qiskit transpilation and pyzx full_reduce were evaluated on the full circuit. For qiskit transpilation we chose optimization level 3, the highest available.