AlphaRouter: Quantum Circuit Routing with Reinforcement Learning and Tree Search

TL;DR

This paper introduces a solution that integrates Monte Carlo Tree Search (MCTS) with Reinforcement Learning (RL) and generates quantum programs with up to 20% less routing overhead, thus significantly enhancing the overall efficiency and feasibility of quantum computing.

Abstract

Quantum computers have the potential to outperform classical computers in important tasks such as optimization and number factoring. They are characterized by limited connectivity, which necessitates the routing of their computational bits, known as qubits, to specific locations during program execution to carry out quantum operations. Traditionally, the NP-hard optimization problem of minimizing the routing overhead has been addressed through sub-optimal rule-based routing techniques with inherent human biases embedded within the cost function design. This paper introduces a solution that integrates Monte Carlo Tree Search (MCTS) with Reinforcement Learning (RL). Our RL-based router, called AlphaRouter, outperforms the current state-of-the-art routing methods and generates quantum programs with up to $20\%$ less routing overhead, thus significantly enhancing the overall efficiency and feasibility of quantum computing.

Figures (11)

• Figure 1: Example of routing a logical circuit to a quantum computer using two SWAPs. Figures \ref{['fig:remaining_circuit_1']}, \ref{['fig:remaining_circuit_2']} show the logical circuits. Colors represent the flow of qubits through the gates. Figures \ref{['fig:qpu_1']}, \ref{['fig:qpu_2']} show the topology and qubit mapping of a ring quantum computer. $p_i,\forall i\in\{0,\ldots,4\}$ represent physical qubits. $l_i,\forall i\in\{0,\ldots,4\}$ represent the logical qubit mapped to a physical qubit. The black edges represent the connections. Figures \ref{['fig:output_1']}, \ref{['fig:output_2']} represent the routed physical circuit output. The gates are scheduled on their target logical qubits and topology compliant.
• Figure 2: A standard RL framework. An agent interacts with an environment to generate and learn from its experiences.
• Figure 3: MCTS with a transformer agent. The process repeats the four stages for a pre-determined max number of iterations. Circles in the tree represent states. Lines connecting the states represent actions.
• Figure 4: AlphaRouter: MCTS + RL training framework.
• Figure 5: AlphaRouter compiles quantum circuits with its trained agent via model inference without MCTS.