Critical States Preparation With Deep Reinforcement Learning

TL;DR

This investigation provides a powerful new framework for preparing and manipulating quantum critical states and focuses on the quantum Rabi model, which can be readily extended to other quantum critical systems described by light-matter interaction models, such as quantum Dicke model.

Abstract

The fast and efficient preparation of quantum critical states is a challenging yet crucial task for various quantum technologies. This difficulty is most particularly for systems near a quantum phase transition, where the closure of the energy gap fundamentally limits the timescale of adiabatic processes and thus precludes rapid state preparation. We propose a framework using deep reinforcement learning (DRL) to rapidly prepare quantum critical states, with broad extendibility to light-matter interaction systems. Specifically, a DRL agent optimizes a set of time-dependent control Hamiltonians to drive the system from an initial noncritical state to a target critical state within a finite time and over experimentally accessible parameter ranges. As a concrete application, we focus on the quantum Rabi model. The DRL-optimized time-dependent control Hamiltonian yield a final state with high-fidelity ($>0.999$) to the target critical state. The protocol can be readily extended to other quantum critical systems described by light-matter interaction models, such as quantum Dicke model. This investigation provides a powerful new framework for preparing and manipulating quantum critical states.

Figures (6)

• Figure 1: Schematic of the DRL approach for quantum critical state preparation. At each episode $n$, the DRL agent obtains the current state $s_{n}$ of the quantum system and the reward $r_{n+1}$, defined by the fidelity between the driven final state $|\Phi(d)\rangle$ and the target state $|\Phi(t)\rangle$ (such as photon--atom, photon--phonon, or magnon systems). Using its policy function $\pi(A|S)$, the agent selects an action $a_{n}$ that specifies a configuration of time-dependent control fields. The selected control is then applied to the quantum system, driving its evolution. This control is applied to the system, driving its evolution to a new state $s_{n+1}$ and yielding a new reward $r_{n+1}$. This iterative loop enables the DRL agent to optimize the control Hamiltonians, guiding the system to the target critical state.
• Figure 2: (a) Energy spectrum $(E_n - E_0)$ of the Rabi Hamiltonian $H_{\rm Rabi}[g(t)]$ across the critical point, calculated with a Hilbert-space truncation dimension of $N = 5000$. (b) Analysis of the trajectory similarity $\Delta_i$, indicating that the control field $(a + a^{\dagger})^2$ plays a dominant role in the system evolution. (c) Waveform and amplitude of the control pulses identified by the DRL-agent. The control Hamiltonian is given by $H^{\rm{c}}_{2}(t) = f_2(t)(a + a^{\dagger})^2$, with the driving field $f_2(t) = \varLambda_{2} \cos(\omega_d t + \phi_2)$. The parameters used in the simulation are $\omega = 1$, $\Omega = 10^4\omega$, $g_0 = 0.01$, $g_{\rm{c}} = 1$, and $K = 60$.
• Figure 3: Time evolution of the populations. At each time $t$, the evolving state $|\Phi(t)\rangle$ is projected onto the first ten eigenstates $|E_{n}(t)\rangle$ of the instantaneous Hamiltonian $H_{\rm{Rabi}}[g(t)]$, yielding the population distribution $P_{n}(t)=|\langle E_{n}(t)|\Phi(t)\rangle|^2$. We also show the Wigner functions of the initial and final states of the cavity subspace, obtained by tracing out the qubit degrees of freedom.
• Figure 4: (a) Analysis of control errors. Different broken lines correspond to errors in different control parameters $\chi$. The results show that the maximum fidelity loss of the final evolved state due to systematic errors remains below $5\%$. (b) Analysis of dissipative effects. The dephasing rate of the qubit is fixed at $\kappa_3/\omega = 0.01$. The results indicate that qubit relaxation has a negligible impact on the final evolved state, and even at relatively large dissipation rates, the fidelity between the final state and the target state remains above $0.99$.
• Figure 5: Time evolution of $I_g$ calculated for the evolving state $|\Phi(t)\rangle$ under the DRL-optimized control protocol, together with $I_s={g^2}/{2(1-g^2)^2}$ in the limit $\omega/\Omega \to 0$, shown as the standard upper bound. The results demonstrate that as the evolution approaches the final time ($t \to T$), the final state acquires extreme sensitivity to the parameter $g$ and $I_g$ approaches $I_s$. Here $\delta = 10^{-5}$ and $g_{s}=0.9999$.