Reinforcement Learning for Adaptive Composition of Quantum Circuit Optimisation Passes

TL;DR

This work addresses the suboptimal reliance on general-purpose default optimisation-pass sequences for quantum circuits by training an RL agent to compose bespoke sequences that reduce two-qubit gate counts. Using PPO with graph neural networks on a circuit-graph representation, the agent selects PyTKET Passes and a DoNothing option to iteratively refine circuits, achieving a cumulative two-qubit gate reduction of $0.577$ mean and $0.567$ median on a diverse test set, outperforming the best default PyTKET sequences. The approach demonstrates strong in-distribution performance, strong generalisation to larger circuits, and superior scalability compared with beam-search and other search-based methods. These results reduce the need for extensive expert tuning and suggest pathways to extending the framework to regional rewrites and broader pass libraries, with potential applicability to other resource metrics such as T-gate counts in fault-tolerant regimes. $\,$

Abstract

Many quantum software development kits provide a suite of circuit optimisation passes. These passes have been highly optimised and tested in isolation. However, the order in which they are applied is left to the user, or else defined in general-purpose default pass sequences. While general-purpose sequences miss opportunities for optimisation which are particular to individual circuits, designing pass sequences bespoke to particular circuits requires exceptional knowledge about quantum circuit design and optimisation. Here we propose and demonstrate training a reinforcement learning agent to compose optimisation-pass sequences. In particular the agent's action space consists of passes for two-qubit gate count reduction used in default PyTKET pass sequences. For the circuits in our diverse test set, the (mean, median) fraction of two-qubit gates removed by the agent is $(57.7\%, \ 56.7 \%)$, compared to $(41.8 \%, \ 50.0 \%)$ for the next best default pass sequence.

Figures (7)

• Figure 1: Training and deploying an RL agent for optimisation-pass selection as described in this work.\ref{['fig:overview training']}: A quantum circuit is loaded into the environment and transformed into a graph representation, as discussed in \ref{['sec:graph representation']}. This graph (the RL "observation" stable-baselines3) is processed by a pair of GNNs, G, as discussed in \ref{['sec:training_methodology']}. The actor network outputs a vector of logits, a, for selecting the next optimisation pass; the critic head estimates the current circuit state value, V. An optimisation pass, as discussed in \ref{['sec:action space']}, is selected via softmax sampling of a and applied to the circuit. The reward, along with V and a, is used to compute gradients, $\nabla$, and update G via PPO. The environment is reset with a fresh circuit from the training dataset (discussed in \ref{['sec:training data']}) whenever either: the DoNothing pass is selected; the two-qubit gate count is reduced to 0; or a preset number of passes are applied with no observed improvement. \ref{['fig:overview deployment']}: An input circuit is converted to a graph and repeatedly ingested by the actor network of G. Optimisation passes are selected via argmax sampling of a. The cycle terminates when the DoNothing pass is selected, and the optimised circuit is returned to the user.
• Figure 2: Example circuit encoded as a graph.
• Figure 3: Cumulative reward (fraction of two-qubit gates removed) for in-distribution circuits.$40,412$ circuits are optimised with nine trained models and a selection of PyTKET optimisation passes. Box plots give the median and interquartile range. Whiskers extend to the the lowest (highest) data point still within $1.5$ times the interquartile range of the lower (upper) quartile. 'Full test dataset' accumulates all of the circuits.
• Figure 4: Difference in cumulative reward (fraction of two-qubit gates removed) between the RL model and baseline optimisation passes for in-distribution circuits. Positive values indicate improved performance of the RL model. 40,412 circuits are optimised with nine trained models and a selection of PyTKET optimisation methods. Box plots give the median and interquartile range. Whiskers extend to the the lowest (highest) data point still within $1.5$ times the interquartile range of the lower (upper) quartile. 'Full test dataset' accumulates all of the circuits. 'Best Baseline' corresponds to the highest cumulative reward generated by any baseline approach for each circuit.
• Figure 5: Cumulative reward (fraction of two-qubit gates removed) for out-of-distribution circuits. 8,115 circuits are optimised with one trained model and a selection of PyTKET optimisation passes. Box plots give the median and interquartile range. Whiskers extend to the the lowest (highest) data point still within $1.5$ times the interquartile range of the lower (upper) quartile. 'Full test dataset' accumulates all of the circuits.