Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The pros and cons of using deep reinforcement learning or genetic algorithms to design control schemes for quantum state transfer on qubit chains

Sofía Perón Santana, Ariel Fiuri, Martín Domínguez, Omar Osenda

TL;DR

This work compares two optimization strategies for quantum state transfer along qubit chains governed by the XX Hamiltonian: a Genetic Algorithm (GA) and a Deep Reinforcement Learning (DRL) approach using a Deep Q-Network (DQN). The GA searches for full control sequences that maximize the transmission probability $P(t)$ and the associated fidelity, achieving high-fidelity transfers at times near the quantum speed limit (QSL) and displaying robustness to weak dynamical noise up to chain lengths around $N=128$. In contrast, the DRL method attains robustness to noise only when trained in noisy environments but generally yields inferior fidelities for longer chains and incurs substantially longer training times. The study concludes that GA-based control is often superior for fast, high-fidelity quantum state transfer in medium-to-large qubit chains, while DRL requires methodological advances (e.g., better exploration and multi-step reward structures) to match GA performance in noisy, open quantum settings, informing method choice for quantum control tasks.

Abstract

In recent years, control methods based on different optimization techniques have shed light on the possibilities of processing information in many quantum systems. When exploring the transmission of quantum states, faster transmission times are mandatory to avoid the deleterious effects of multiple sources of decoherence that spoil the transmission process. In particular, using Reinforcement Learning to devise sequences of step-wise external controls provides good transfer policies at short transmission times. We present two approaches to control the transmission of quantum states in qubit chains using external controls to force the dynamical evolution of the chain state. The first approach relies on the well-known Genetic Algorithm to generate a sequence of external controls, while the second approach uses a variant of Reinforcement Learning. The Genetic algorithm achieves excellent transmission fidelity at as short transmission times as Reinforcement Learning, surpassing the fidelities achieved by the latter method. Nevertheless, the Reinforcement Learning method offers robust control policies when the control pulses are noisy enough, owing to an imperfect timing of the pulses, deficient control devices, or other sources of phase decoherence. We present the regime where each method is best suited to control the transmission of arbitrary qubit states.

The pros and cons of using deep reinforcement learning or genetic algorithms to design control schemes for quantum state transfer on qubit chains

TL;DR

This work compares two optimization strategies for quantum state transfer along qubit chains governed by the XX Hamiltonian: a Genetic Algorithm (GA) and a Deep Reinforcement Learning (DRL) approach using a Deep Q-Network (DQN). The GA searches for full control sequences that maximize the transmission probability and the associated fidelity, achieving high-fidelity transfers at times near the quantum speed limit (QSL) and displaying robustness to weak dynamical noise up to chain lengths around . In contrast, the DRL method attains robustness to noise only when trained in noisy environments but generally yields inferior fidelities for longer chains and incurs substantially longer training times. The study concludes that GA-based control is often superior for fast, high-fidelity quantum state transfer in medium-to-large qubit chains, while DRL requires methodological advances (e.g., better exploration and multi-step reward structures) to match GA performance in noisy, open quantum settings, informing method choice for quantum control tasks.

Abstract

In recent years, control methods based on different optimization techniques have shed light on the possibilities of processing information in many quantum systems. When exploring the transmission of quantum states, faster transmission times are mandatory to avoid the deleterious effects of multiple sources of decoherence that spoil the transmission process. In particular, using Reinforcement Learning to devise sequences of step-wise external controls provides good transfer policies at short transmission times. We present two approaches to control the transmission of quantum states in qubit chains using external controls to force the dynamical evolution of the chain state. The first approach relies on the well-known Genetic Algorithm to generate a sequence of external controls, while the second approach uses a variant of Reinforcement Learning. The Genetic algorithm achieves excellent transmission fidelity at as short transmission times as Reinforcement Learning, surpassing the fidelities achieved by the latter method. Nevertheless, the Reinforcement Learning method offers robust control policies when the control pulses are noisy enough, owing to an imperfect timing of the pulses, deficient control devices, or other sources of phase decoherence. We present the regime where each method is best suited to control the transmission of arbitrary qubit states.
Paper Structure (10 sections, 15 equations, 11 figures, 3 tables)

This paper contains 10 sections, 15 equations, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Model and methods
  3. Control policies designed using Genetic Algorithm
  4. Control policies designed using Reinforcement Learning
  5. Robustness against noisy control pulses
  6. Discussion and conclusions
  7. Details about the Genetic Algorithm
  8. RL algorithm hyperparameters and definitions
  9. Action distribution for genetic algorithm solutions.
  10. Performance of genetic algorithm for larger qubit chains

Figures (11)

  • Figure 1: The cartoon in the figure depicts a system of $N$ qubits and its time evolution. The initial state, shown at the leftmost extreme of the cartoon, corresponds to a one-excitation quantum state. The step-wise evolution operator for a given interval, $U_k = U(\tau_k)$, acts over all the qubits and is shown as a gate that applies over the whole system. The specific form of each evolution operator depends on the controls operating on the corresponding time interval.
  • Figure 2: The cartoon in the Figure presents the main ingredients of the Genetic Algorithm. a) The sixteen possible actions, each of which can appear on a control sequence at any position in it. b) An initial population of four individuals, each one endowed with its own chromosome. The chromosome contains the genes, each one an action. c) For each sequence $c_i$ (or chromosome), the algorithm calculates its fitness $f_i$, which corresponds to the maximum value attained by the transmission probability along the time interval $\sum_k \tau_k$. d) The algorithm selects a set of fittest individuals (in this case, two parents). e) and f) the chromosomes of the new members of the population result from the application of two rules, uniform crossover and swap mutation. g) The new population contains the set of parents and their offspring.
  • Figure 3: The figure shows the maximum value of transmission probability achieved by the genetic algorithm for a chain of $N=64$ spins for different values of $h$ and $\Delta t$.
  • Figure 4: The figure summarises the effect of the initial conditions on the values obtained for the transmission probability. The GA is a biased random search algorithm; consequently, it is always advisable to study the variability of the results obtained. Panel a) shows the maximum value obtained for the transmission probability considering $30$ different initial populations. The solid dots correspond to the values of the transmission probability, the circular dots correspond to the maximum over all the realisations of the initial populations, and the square dots to the value obtained by averaging the maxima of the realisations. The site-by-site action set results are much stable than the action set with a fixed number of elements. Note the different sizes of the bar errors, which correspond to one standard deviation. We obtained the data points by imposing convergence criteria, by asking each algorithm to halt after the transmission probability reaches a target figure of $0.99$, after a total of $1000$ generations or after $30$ generations without an increment in fitness. Panel b) shows the time evolution obtained using the control sequences whose transmission probabilities appear in panel a).
  • Figure 5: The figure shows (a) the maximum transmission probabilities obtained using the DQN algorithm for spin chains of different lengths, and (b) the transmission probability as a function of time for a chain of 32 qubits. The action set used to obtain the data is the one proposed by Zhang and coworkers. For the values of the hyperparameters, see the Table \ref{['tab:original-hyperparams']}. While the initial state of the transmission problem is the same in all cases, that is not the case with the initial weights of the Q and target networks, even when the algorithm’s hyperparameters are kept unchanged. To test the robustness of the method, the algorithm randomly sorts the initial weights. Panel (a) presents results from 30 independent network realisations, reporting both the highest transmission probability achieved across all realisations and the average of the maximum transmission probabilities obtained in each realisation. The circular dots correspond to the averaged maximum probabilities, and the circular dots to the maximum obtained over all the realisations. The error bars correspond to the standard deviation of the data. Panel b) shows the transmission probability vs time. The black lines correspond to the forced evolution driven by the actions that achieve the maxima for the different network initial conditions, whose average and best value we plot in panel a). The green line corresponds to the autonomous, unforced time-evolution.
  • ...and 6 more figures