QAS-QTNs: Curriculum Reinforcement Learning-Driven Quantum Architecture Search for Quantum Tensor Networks

TL;DR

The paper addresses automated quantum circuit design for variational quantum circuits using Quantum Architecture Search (QAS) combined with Quantum Tensor Networks (QTNs). It introduces QAS-QTNs, a hybrid classical-quantum reinforcement learning framework augmented by quantum curriculum learning and Prioritized Experience Replay to tackle progressively harder circuit design tasks. Empirical results show quantum-enhanced RL (e.g., PERQDDQN, PERQTD3) achieving higher success probabilities and more optimal circuit configurations than classical RL in 2- and 3-qubit Bell/GHZ state preparation, and a real-world Iris classification task reaching 90.33% accuracy. The work demonstrates a scalable approach to automated quantum architecture discovery with potential impact on quantum ML and hardware-aware circuit design.

Abstract

Quantum Architecture Search (QAS) is an emerging field aimed at automating the design of quantum circuits for optimal performance. This paper introduces a novel QAS framework employing hybrid quantum reinforcement learning with quantum curriculum learning strategies, enabling learning agents to tackle increasingly complex quantum circuit design tasks. We benchmark four state-of-the-art classical reinforcement learning algorithms (A2C, PPO, DDQN, TD3) against their quantum-enhanced counterparts (QA2C, QPPO, QDDQN, QTD3) for optimizing variational quantum circuits (VQCs). Our approach progressively increases circuit depth and gate complexity during training, leveraging parameterized quantum circuits as function approximations. To improve learning efficiency and stability, all algorithms, both classical and quantum, are augmented with Prioritized Experience Replay (PER). Experimental results show that quantum-enhanced RL significantly outperforms classical methods. In a 2-qubit environment, PERQDDQN achieves a success probability of 0.46 with ~3,000 optimal successes, surpassing classical PERDDQN (0.42, ~2,400). In the more complex 3-qubit setting, PERQDDQN and PERQTD3 reach success probabilities of ~0.47, with optimal success counts of ~3,800 and ~3,600, respectively, outperforming their classical counterparts. Additionally, we apply our QAS-QTN approach to a classification problem, where the optimized quantum circuit achieves an accuracy of 90.33\%, outperforming quantum models consisting of random ansatz. This hybrid classical-quantum approach leads to faster convergence and more efficient quantum circuit designs, demonstrating its potential for advancing automated quantum architecture search.

Figures (3)

• Figure 1: The proposed workflow illustrates the hybrid QRL framework for QAS, integrating curriculum learning to progressively optimize VQCs. The agent interacts with a quantum circuit optimization environment, sampling QTNs components while refining policies through RL. A replay buffer stores experience transitions to enhance learning, and circuit difficulty adapts dynamically based on fidelity evaluations, ensuring convergence to optimal quantum states.
• Figure 2: Quantum circuits for entanglement generation: (left) a Bell state ($|\Phi^+\rangle$) is created by applying a Hadamard gate to the first qubit, followed by a CNOT gate; (right) a GHZ state is generated by extending the Bell state with a second CNOT gate to entangle a third qubit.
• Figure 3: Comparison of RL algorithms in quantum environments, the plots illustrate the performance of multiple RL algorithms in constructing Bell and GHZ states across 2-qubit and 3-qubit quantum environments. Success Probability and Optimal Success Count metrics are tracked over 10,000 training steps. Hybrid quantum algorithms, such as PERQDDQN & PERQTD3, exhibit superior learning efficiency and higher success rates in both environments, with the differences in performance becoming more pronounced in the more complex 3-qubit setting, highlighting the scalability and effectiveness of prioritization-based strategies in hybrid quantum RL tasks.