Reinforcement Learning for Quantum Technology

TL;DR

The paper surveys how reinforcement learning addresses core challenges in quantum technology, from state preparation and gate design to circuit synthesis, feedback control, and metrology. It outlines formal RL frameworks tailored to quantum systems, contrasts model‑free and model‑based approaches, and highlights a growing set of experimental demonstrations and open challenges. Key contributions include integrating RL with quantum optimal control, presenting diverse applications across few‑ and many‑body systems, and identifying practical hurdles such as scalability, partial observability, and hardware integration. The work underscores RL’s potential to autonomously optimize quantum protocols, enable adaptive control under noise, and catalyze progress toward robust, scalable quantum technologies.

Abstract

Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.

Figures (12)

• Figure 1: Reinforcement Learning (RL) is a framework of (deep) learning algorithms that can be used for interactive feedback control of quantum devices. Illustration created in part using Google Gemini.
• Figure 2: Illustration of the basic principle of reinforcement learning. A puppy is trained to fetch a stick. At each new attempt, the puppy decides whether to fetch or not. As a consequence of the chosen action, its environment responds and changes its state. A treat is given to the puppy in the form of a reward to incentivize the desired behavior. Illustrations created using OpenAI’s DALL·E model.
• Figure 3: Training process in RL. RL agents are usually trained in episodic tasks, where each training iteration comprises an episode, which consists of discrete steps. At each step, the agent selects an action that changes the state of the environment. The goal for the agent is, based on the observation of the current environment state, to select that action which maximizes the sum of the rewards within the episode.
• Figure 4: Overview of RL algorithms. RL algorithms fall into two overlapping categories. Policy gradient methods have the primary objective to improve the policy $\pi$. Value-function methods, on the other hand, learn a value function (e.g., the value function $v_\pi$, the Q-function $Q_\pi$, or the advantage function $A_\pi$). Actor-critic methods are a hybrid class of algorithms that learn both a policy and a value function.
• Figure 5: Common deep learning architectures used in RL. (a) The RL policy $\pi_\theta(a|s)$ corresponds to a histogram over the actions for a discrete action space, for each input state $s$. (b) For continuous actions, the policy ansatz learns the state-dependent parameters of a continuous probability density (e.g., the mean $\mu(s)$ and variance $\sigma^2(s)$ of a Gaussian). (c) A typical architecture for the $Q$-function has RL states as input, and outputs the $Q_{\theta,\pi}(s,a)$ value for each of the actions. (d) Value functions $v_{\theta,\pi}(s)$ take in states $s$ and output a single scalar.