Harnessing Environmental Memory with Reinforcement Learning in Open Quantum Systems

TL;DR

The paper addresses harnessing non-Markovian memory in open quantum systems by maximizing the Breuer-Laine-Piilo (BLP) non-Markovianity measure through model-free reinforcement learning. It models a driven two-level system coupled to a Lorentzian bath, using a time-dependent decay rate $\gamma(t)$ that can become negative, enabling information backflow. The authors compare two RL agents, SAC and PPO, against gradient-based OCT, finding that RL redistributes backflow over multiple memory windows to achieve a larger integrated non-Markovianity $N_{Tot}$ (with PPO ~$0.37$ and SAC ~$0.29$) whereas OCT concentrates on a single revival, yielding a smaller $N_{Tot}$. This demonstrates a practical, data-driven route to engineering memory effects in open quantum systems and suggests RL's potential for experimental deployment and extension to more complex, memory-rich quantum tasks.

Abstract

Non-Markovian memory effects in open quantum systems provide valuable resources for preserving coherence and enhancing controllability. However, exploiting them requires strategies adapted to history-dependent dynamics. We introduce a reinforcement-learning framework that autonomously learns to amplify information backflow in a driven two-level system coupled to a structured reservoir. Using a reward based on the positive time derivative of the trace distance associated with the Breuer-Laine-Piilo measure, we train PPO and SAC agents and benchmark their performance against gradient-based optimal control theory (OCT). While OCT enhances a single dominant backflow peak, RL policies broaden this revival and activate additional contributions in later memory windows, producing sustained positive trace-distance growth over a longer duration. Consequently, the integrated non-Markovianity achieved by RL substantially exceeds that obtained with OCT. These results demonstrate how long-horizon, model-free learning naturally uncovers distributed-backflow strategies and highlight the potential of reinforcement learning for engineering memory effects in open quantum systems.

Figures (12)

• Figure 1: Time-dependent decay rate $\gamma(t)$ for a Lorentzian spectral density in the strong-coupling regime $\Gamma>\lambda/2$. The regions where $\gamma(t)<0$ (shaded in red) correspond to non-Markovian intervals where information flows back into the system.
• Figure 2: Convergence history of the total non-Markovianity $\mathcal{N}_{Tol}$ for Powell (red) and LBFGSB (blue). Both algorithms improve $\mathcal{N}_{Tol}$ relative to the uncontrolled case, but rapidly plateau at different local optima, indicating the presence of a strongly non-convex control landscape shaped by memory effects.
• Figure 3: Instantaneous non-Markovianity $\mathcal{N}_{\rm loc}(t)$ for the uncontrolled dynamics and after OCT optimization with Powell (red) and L-BFGS-B (blue), illustrating modified backflow features under control.
• Figure 4: Optimized control pulses $\Omega(t)$ obtained with Powell (top) and LBFGSB (bottom).The two methods converge to distinct local optima, consistent with a fragmented control landscape in the presence of non-Markovian effects.
• Figure 5: Reinforcement-learning control loop for maximizing non-Markovianity. The agent observes $(D,\dot D,\gamma,\Omega)$, selects actions, and receives rewards based on information backflow.