Reinforcement Learning for Quantum Circuit Design: Using Matrix Representations

TL;DR

Addressing automated quantum circuit design in the NISQ era, the paper frames circuit search as Markov Decision Processes and applies reinforcement learning to synthesize gate sequences. It introduces three MDP representations—Matrix Representation, Reverse Matrix Representation, and Tensor Network Representation—and demonstrates Q-learning and Deep Q-Network (DQN) methods to reach target circuits under the gate set $\{H, T, \text{CNOT}\}$. For a Bell-state target, e.g., $|\Phi^+\rangle=\frac{1}{\sqrt{2}}(|00\rangle+|11\rangle)$ with $U=\text{CNOT}_{01}(H\otimes I)$, the RL agents learn sequences that realize $U|00\rangle=|\Phi^+\rangle$, as shown across representations. The results indicate success on multiple tasks but highlight scalability challenges with larger state spaces, pointing to future improvements like Monte Carlo Tree Search and enhanced trajectory sampling.

Abstract

Quantum computing promises advantages over classical computing. The manufacturing of quantum hardware is in the infancy stage, called the Noisy Intermediate-Scale Quantum (NISQ) era. A major challenge is automated quantum circuit design that map a quantum circuit to gates in a universal gate set. In this paper, we present a generic MDP modeling and employ Q-learning and DQN algorithms for quantum circuit design. By leveraging the power of deep reinforcement learning, we aim to provide an automatic and scalable approach over traditional hand-crafted heuristic methods.

Figures (15)

• Figure 1: A quantum circuit to generate Bell state $\ket{\Phi^+}$.
• Figure 2: State tree in matrix representation for searching the circuit in Fig. \ref{['fig:bell_state_0']}.
• Figure 3: State tree in reverse matrix representation for searching the circuit in Fig. \ref{['fig:bell_state_0']}.
• Figure 4: TN representation of Fig. \ref{['fig:bell_state_0']}.
• Figure 5: State tree in TN representation for searching the circuit in Fig. \ref{['fig:bell_state_0']}.