Reinforcement learning-assisted quantum architecture search for variational quantum algorithms

TL;DR

This thesis tackles the issue of noise on the QAS by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network with an $\epsilon$-greedy policy for better stability.

Abstract

A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an $ε$-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.

Figures (49)

• Figure 1: The workflow of variational quantum algorithms (VQAs). The process starts with an input quantum state, which is processed through a series of one- and two-qubit gates within the parametric quantum circuit (PQC). The output is the evolved quantum state from the PQC. This contrasts with a classical subroutine that optimizes the PQC parameters using classical optimization methods to minimize a cost function encoding a problem. The optimized parameters are then fed back to update the PQC iteratively until the problem is solved.
• Figure 2: An illustration of the cost function landscape for the variational quantum state diagonalization algorithm larose2019variational with a two-qubit mixed quantum state. As an ansatz, we choose a layer of rotation RY and RZ on both the qubits, followed by a CNOT with control on the first qubit.
• Figure 3: Exponential decay of the variance of the cost function gradient for quantum state diagonalization problem for three-qubit of depth 2 ansatz.
• Figure 4: Illustration of the variation of the number of CNOT gates and depth of the circuit with increasing spin orbitals in LiH molecule with UCCSD ansatz.
• Figure 5: An example of HEA depending on the topology of IBM quantum devices. In (a), we present HEA that follows the topology of ibmq_manila whereas in (b) the HEA follows the topology of ibm_quito, ibm_belem and ibm_lima. It should be noted that the $G_i$ are parametrized, and Ent is the entangling unitary. The quantum wire through the last Ent gate means the gate does not apply on that qubit.

Theorems & Definitions (2)

• Definition 3.1.1
• Proposition 1