Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Variational Quantum Circuits Design

Simone Foderà, Gloria Turati, Riccardo Nembrini, Maurizio Ferrari Dacrema, Paolo Cremonesi

TL;DR

The paper tackles the challenge of designing effective variational quantum circuit ansatzes for optimization on NISQ devices by training a reinforcement learning agent (RLVQC) to autonomously construct circuits. It evaluates the agent on QUBO-formulated problems (Maximum Cut, Maximum Clique, Minimum Vertex Cover) across multiple graph topologies, uncovering a novel $R_{yz}$-connected ansatz, with a representative Linear circuit performing well on Maximum Cut. Compared to QAOA variants, RLVQC achieves comparable or superior approximation ratios in several instances, and the $R_{yz}$-connected family demonstrates hardware-friendly properties, including decomposability and reduced SWAP overhead. The work highlights RL as a promising route to automate quantum circuit design, with potential extensions to hardware-aware customization and broader QC tasks.

Abstract

Variational Quantum Algorithms have emerged as promising tools for solving optimization problems on quantum computers. These algorithms leverage a parametric quantum circuit called ansatz, where its parameters are adjusted by a classical optimizer with the goal of optimizing a certain cost function. However, a significant challenge lies in designing effective circuits for addressing specific problems. In this study, we leverage the powerful and flexible Reinforcement Learning paradigm to train an agent capable of autonomously generating quantum circuits that can be used as ansatzes in variational algorithms to solve optimization problems. The agent is trained on diverse problem instances, including Maximum Cut, Maximum Clique and Minimum Vertex Cover, built from different graph topologies and sizes. Our analysis of the circuits generated by the agent and the corresponding solutions shows that the proposed method is able to generate effective ansatzes. While our goal is not to propose any new specific ansatz, we observe how the agent has discovered a novel family of ansatzes effective for Maximum Cut problems, which we call $R_{yz}$-connected. We study the characteristics of one of these ansatzes by comparing it against state-of-the-art quantum algorithms across instances of varying graph topologies, sizes, and problem types. Our results indicate that the $R_{yz}$-connected circuit achieves high approximation ratios for Maximum Cut problems, further validating our proposed agent. In conclusion, our study highlights the potential of Reinforcement Learning techniques in assisting researchers to design effective quantum circuits which could have applications in a wide number of tasks.

Reinforcement Learning for Variational Quantum Circuits Design

TL;DR

The paper tackles the challenge of designing effective variational quantum circuit ansatzes for optimization on NISQ devices by training a reinforcement learning agent (RLVQC) to autonomously construct circuits. It evaluates the agent on QUBO-formulated problems (Maximum Cut, Maximum Clique, Minimum Vertex Cover) across multiple graph topologies, uncovering a novel -connected ansatz, with a representative Linear circuit performing well on Maximum Cut. Compared to QAOA variants, RLVQC achieves comparable or superior approximation ratios in several instances, and the -connected family demonstrates hardware-friendly properties, including decomposability and reduced SWAP overhead. The work highlights RL as a promising route to automate quantum circuit design, with potential extensions to hardware-aware customization and broader QC tasks.

Abstract

Variational Quantum Algorithms have emerged as promising tools for solving optimization problems on quantum computers. These algorithms leverage a parametric quantum circuit called ansatz, where its parameters are adjusted by a classical optimizer with the goal of optimizing a certain cost function. However, a significant challenge lies in designing effective circuits for addressing specific problems. In this study, we leverage the powerful and flexible Reinforcement Learning paradigm to train an agent capable of autonomously generating quantum circuits that can be used as ansatzes in variational algorithms to solve optimization problems. The agent is trained on diverse problem instances, including Maximum Cut, Maximum Clique and Minimum Vertex Cover, built from different graph topologies and sizes. Our analysis of the circuits generated by the agent and the corresponding solutions shows that the proposed method is able to generate effective ansatzes. While our goal is not to propose any new specific ansatz, we observe how the agent has discovered a novel family of ansatzes effective for Maximum Cut problems, which we call -connected. We study the characteristics of one of these ansatzes by comparing it against state-of-the-art quantum algorithms across instances of varying graph topologies, sizes, and problem types. Our results indicate that the -connected circuit achieves high approximation ratios for Maximum Cut problems, further validating our proposed agent. In conclusion, our study highlights the potential of Reinforcement Learning techniques in assisting researchers to design effective quantum circuits which could have applications in a wide number of tasks.
Paper Structure (14 sections, 13 equations, 4 figures, 1 table)

This paper contains 14 sections, 13 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Variational Quantum Algorithms
  3. Reinforcement Learning
  4. Reinforcement Learning for Quantum Computing
  5. Agent Design and Training
  6. Environment, Actions, Reward
  7. Training Details
  8. Problem Instances
  9. Evaluation Metrics
  10. Results and Discussion
  11. RLVQC Results
  12. $R_{yz}$-connected Ansatzes
  13. Implementability of $R_{yz}$-connected Ansatzes
  14. Conclusions and Future Directions

Figures (4)

  • Figure 1: Visual overview of the RL pipeline, describing the interaction between agent and environment (see \ref{['fig:rl:int']}) and how the agent (see \ref{['fig:rl:agent']}) and environment (see \ref{['fig:rl:env']}) work internally.
  • Figure 2: Linear circuit, a specific element of the family of $R_{yz}$-connected ansatzes found during training on Maximum Cut.
  • Figure 3: Approximation Ratio obtained by the tested quantum algorithms on Maximum Cut instances on graphs with 16 vertices.
  • Figure 4: Comparison of solution distributions in circuits with optimal parameters solving the Maximum Cut problem on a $n=16$ vertices Erdős–Rényi graph with edge probability 0.8. Results are obtained using the Linear circuit and QAOA with $p=1$ with their parameter optimized, each executed with 1000 shots. The x-axis displays cost values of the QUBO formulation cost function, while the y-axis shows the frequency of each cost occurrence within the total shots.