Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization

Vincent Gurgul, Ying Chen, Stefan Lessmann

TL;DR

This work investigates dynamic portfolio optimization with quantum reinforcement learning by deploying a fully quantum RL framework based on Variational Quantum Circuits. It encodes market data via an extended amplitude map, uses PQCs for both the actor and critic, and derives portfolio weights from Pauli-Z readouts with gradient updates via the parameter-shift rule. Empirical results on real-world data show that QRL models with only 30–60 trainable parameters can match or exceed the performance of classical Deep RL models that require orders of magnitude more parameters, although end-to-end latency is currently dominated by cloud infrastructure. The study highlights a compelling potential for QRL in high-dimensional, non-stationary finance, while acknowledging that practical deployment awaits reductions in quantum hardware access overheads; the work also provides open-source code for reproducibility.

Abstract

This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public

Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization

TL;DR

This work investigates dynamic portfolio optimization with quantum reinforcement learning by deploying a fully quantum RL framework based on Variational Quantum Circuits. It encodes market data via an extended amplitude map, uses PQCs for both the actor and critic, and derives portfolio weights from Pauli-Z readouts with gradient updates via the parameter-shift rule. Empirical results on real-world data show that QRL models with only 30–60 trainable parameters can match or exceed the performance of classical Deep RL models that require orders of magnitude more parameters, although end-to-end latency is currently dominated by cloud infrastructure. The study highlights a compelling potential for QRL in high-dimensional, non-stationary finance, while acknowledging that practical deployment awaits reductions in quantum hardware access overheads; the work also provides open-source code for reproducibility.

Abstract

This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public
Paper Structure (21 sections, 25 equations, 7 figures, 2 tables, 4 algorithms)

This paper contains 21 sections, 25 equations, 7 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Quantum Portfolio Optimization
  4. Quantum Reinforcement Learning
  5. Identified Gaps
  6. Methodology
  7. Foundations of Quantum Computing
  8. Variational Quantum Circuits and Quantum Neural Networks
  9. Reinforcement Learning
  10. Reinforcement Learning for Portfolio Optimization
  11. Quantum Reinforcement Learning for Portfolio Optimization
  12. Experimental Design
  13. Data and Environment
  14. Objective and Evaluation Metrics
  15. Baselines and Comparisons
  16. ...and 6 more sections

Figures (7)

  • Figure 3.1: Visualization of the Bloch sphere freiman_hydrogen_2024.
  • Figure 3.2: Illustration of the four steps involved in inference with a VQC schuld_circuit-centric_2020-1.
  • Figure 3.3: A PQC with three qubits, nine parameterized single-qubit rotation gates (red) and four two-qubit entangling CNOT gates (blue).
  • Figure 3.4: Visualization of converged VQC with one, three, and five layers of rotation and entanglement gates, illustrating how increasing circuit depth enables the representation as higher-order Fourier series. schuld_effect_2021.
  • Figure 3.5: Architecture of our proposed QRL pipeline for portfolio optimization in the illustrative case of a two-asset, two-time-step environment.
  • ...and 2 more figures