Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial

Abhijit Sen, Sonali Panda, Mahima Arya, Subhajit Patra, Zizhan Zheng, Denys I. Bondar

TL;DR

This paper presents a beginner‑oriented tutorial that demystifies reinforcement learning by using a single coherent example to connect theory with practical coding. It systematically builds RL foundations—from basic concepts and DP to policy‑based methods and actor–critic frameworks—before translating these ideas to quantum control and quantum reinforcement learning. The authors illustrate concrete RL applications to quantum state preparation, gate synthesis, and tracking control, including a high‑fidelity qubit example (F ≈ 1) and discussions of tracking trajectories. By integrating detailed explanations with ready‑to‑use code and outlining future directions in tracking control and quantum‑assisted learning, the work aims to make RL accessible to physics students and practical for hybrid quantum‑classical systems.

Abstract

This tutorial is designed to make reinforcement learning (RL) more accessible to undergraduate students by offering clear, example-driven explanations. It focuses on bridging the gap between RL theory and practical coding applications, addressing common challenges that students face when transitioning from conceptual understanding to implementation. Through hands-on examples and approachable explanations, the tutorial aims to equip students with the foundational skills needed to confidently apply RL techniques in real-world scenarios.

From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial

TL;DR

This paper presents a beginner‑oriented tutorial that demystifies reinforcement learning by using a single coherent example to connect theory with practical coding. It systematically builds RL foundations—from basic concepts and DP to policy‑based methods and actor–critic frameworks—before translating these ideas to quantum control and quantum reinforcement learning. The authors illustrate concrete RL applications to quantum state preparation, gate synthesis, and tracking control, including a high‑fidelity qubit example (F ≈ 1) and discussions of tracking trajectories. By integrating detailed explanations with ready‑to‑use code and outlining future directions in tracking control and quantum‑assisted learning, the work aims to make RL accessible to physics students and practical for hybrid quantum‑classical systems.

Abstract

This tutorial is designed to make reinforcement learning (RL) more accessible to undergraduate students by offering clear, example-driven explanations. It focuses on bridging the gap between RL theory and practical coding applications, addressing common challenges that students face when transitioning from conceptual understanding to implementation. Through hands-on examples and approachable explanations, the tutorial aims to equip students with the foundational skills needed to confidently apply RL techniques in real-world scenarios.
Paper Structure (30 sections, 123 equations, 8 figures, 2 tables)

This paper contains 30 sections, 123 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Understanding RL
  3. Essential Preliminaries
  4. Terminology Alert: Part I
  5. Terminology Alert: Part II
  6. Objective of RL Part I: Setting the Stage
  7. Objective of RL Part II: Policy Evaluation and Policy Improvement
  8. Markov Chain and Markov Decision Process (MDP)
  9. Dynamic Programming (DP)
  10. Monte Carlo Methods
  11. Temporal Difference Methods
  12. Policy Based Methods
  13. Policy Gradient I: The Theory
  14. Policy Gradient II: Example
  15. Actor-Critic Method
  16. ...and 15 more sections

Figures (8)

  • Figure 1: A 1D grid with a robot in cell $s_1$ and a wall in cell $s_5$.
  • Figure 2: An improved version of Figure \ref{['fig:grid_with_robot_and_wall']} with action and rewards explicitly shown.
  • Figure 3: Another version of Fig. \ref{['fig:grid_with_robot_and_wall']} with robot in cell 2 taking first action to left and then following policy $\pi$
  • Figure 4: Updated version of Fig. \ref{['fig:rewardfig']} with improved policy
  • Figure 5: Robot's trajectories from cell s3 to the target cell s7 with different paths represented by colored arrows.
  • ...and 3 more figures