Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Machine Learning Architecture Search via Deep Reinforcement Learning

Xin Dai, Tzu-Chieh Wei, Shinjae Yoo, Samuel Yen-Chi Chen

TL;DR

The paper tackles the challenge of designing quantum machine learning models suitable for Noisy Intermediate-Scale Quantum devices by introducing RL-QMLAS, a deep reinforcement learning framework that autonomously searches for effective variational quantum circuit architectures without a predefined ansatz. It leverages Q-learning variants (DDQN and N-step DDQN) with an adaptive target learning mechanism to optimize a gate-efficient quantum classifier for binary classification tasks, using an arctan-based input embedding and a $4\times L$ state representation. Experimental results on make_classification and make_moons demonstrate that the agent can achieve high classification accuracy with shallow circuits, often using only a small number of gates, and that the adaptive search strategy can further improve learning efficiency and final performance. The work highlights the potential of AI-driven quantum circuit design to reduce the need for quantum expertise and to enable practical QML deployment on current quantum hardware, while outlining challenges related to noise, scalability, and hardware realism.

Abstract

The rapid advancement of quantum computing (QC) and machine learning (ML) has given rise to the burgeoning field of quantum machine learning (QML), aiming to capitalize on the strengths of quantum computing to propel ML forward. Despite its promise, crafting effective QML models necessitates profound expertise to strike a delicate balance between model intricacy and feasibility on Noisy Intermediate-Scale Quantum (NISQ) devices. While complex models offer robust representation capabilities, their extensive circuit depth may impede seamless execution on extant noisy quantum platforms. In this paper, we address this quandary of QML model design by employing deep reinforcement learning to explore proficient QML model architectures tailored for designated supervised learning tasks. Specifically, our methodology involves training an RL agent to devise policies that facilitate the discovery of QML models without predetermined ansatz. Furthermore, we integrate an adaptive mechanism to dynamically adjust the learning objectives, fostering continuous improvement in the agent's learning process. Through extensive numerical simulations, we illustrate the efficacy of our approach within the realm of classification tasks. Our proposed method successfully identifies VQC architectures capable of achieving high classification accuracy while minimizing gate depth. This pioneering approach not only advances the study of AI-driven quantum circuit design but also holds significant promise for enhancing performance in the NISQ era.

Quantum Machine Learning Architecture Search via Deep Reinforcement Learning

TL;DR

The paper tackles the challenge of designing quantum machine learning models suitable for Noisy Intermediate-Scale Quantum devices by introducing RL-QMLAS, a deep reinforcement learning framework that autonomously searches for effective variational quantum circuit architectures without a predefined ansatz. It leverages Q-learning variants (DDQN and N-step DDQN) with an adaptive target learning mechanism to optimize a gate-efficient quantum classifier for binary classification tasks, using an arctan-based input embedding and a state representation. Experimental results on make_classification and make_moons demonstrate that the agent can achieve high classification accuracy with shallow circuits, often using only a small number of gates, and that the adaptive search strategy can further improve learning efficiency and final performance. The work highlights the potential of AI-driven quantum circuit design to reduce the need for quantum expertise and to enable practical QML deployment on current quantum hardware, while outlining challenges related to noise, scalability, and hardware realism.

Abstract

The rapid advancement of quantum computing (QC) and machine learning (ML) has given rise to the burgeoning field of quantum machine learning (QML), aiming to capitalize on the strengths of quantum computing to propel ML forward. Despite its promise, crafting effective QML models necessitates profound expertise to strike a delicate balance between model intricacy and feasibility on Noisy Intermediate-Scale Quantum (NISQ) devices. While complex models offer robust representation capabilities, their extensive circuit depth may impede seamless execution on extant noisy quantum platforms. In this paper, we address this quandary of QML model design by employing deep reinforcement learning to explore proficient QML model architectures tailored for designated supervised learning tasks. Specifically, our methodology involves training an RL agent to devise policies that facilitate the discovery of QML models without predetermined ansatz. Furthermore, we integrate an adaptive mechanism to dynamically adjust the learning objectives, fostering continuous improvement in the agent's learning process. Through extensive numerical simulations, we illustrate the efficacy of our approach within the realm of classification tasks. Our proposed method successfully identifies VQC architectures capable of achieving high classification accuracy while minimizing gate depth. This pioneering approach not only advances the study of AI-driven quantum circuit design but also holds significant promise for enhancing performance in the NISQ era.
Paper Structure (20 sections, 7 equations, 11 figures)

This paper contains 20 sections, 7 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Relevant Works
  3. Variational Quantum Circuits
  4. Quantum Architecture Search
  5. Reinforcement Learning
  6. Q-Learning
  7. Double Deep Q-Learning
  8. N-Step DDQN
  9. Methods
  10. Experimental setup
  11. Hyperparameters of RL
  12. N-step Double Deep Q-Network (DDQN)
  13. Reward Function
  14. Adaptive Search
  15. Results
  16. ...and 5 more sections

Figures (11)

  • Figure 1: Overall scheme for RL-QMLAS.
  • Figure 2: Generic variational quantum circuit (VQC) structure.
  • Figure 3: Reinforcement learning performance metrics using the make_classification dataset with a fixed target accuracy of 0.85 and a maximum of 20 quantum gates for a total of 800 episodes. Training accuracy (a) and the number of gates (c) are smoothed with a 40-episode moving average. For testing accuracy (b) and gate count (d), a 4-episode moving average is applied. The maximum training epoch after each action is set to 15. The rewards patterns during training (e) and testing (f) further demonstrate the agent's learning.
  • Figure 4: Reinforcement learning performance metrics using the make_moons dataset with a target accuracy of 0.85 and a maximum of 25 quantum gates for a total of 800 episodes. Training accuracy (a) and the number of gates (c) are smoothed with a 40-episode moving average. For testing accuracy (b) and gate count (d), a 4-episode moving average is applied. The maximum training epoch after each action is set to 25.
  • Figure 5: Performance of the reinforcement learning agent on the make_classification dataset over 1200 episodes using a adaptive search strategy, starting from an initial target accuracy of 0.8. Panel (a) includes the dynamic target accuracy, which is adjusted by the adaptive search strategy, alongside the training accuracy. All other experimental conditions and metric smoothing methods align with those described for the fixed target experiments.
  • ...and 6 more figures

Theorems & Definitions (1)

  • Definition 4.1: QAS for Quantum Supervised Learning