Reinforcement learning-based architecture search for quantum machine learning

TL;DR

The paper tackles the challenge of designing problem-specific encoding circuits for quantum machine learning by framing circuit generation as a model-based reinforcement learning problem. Using MuZero, it builds a layered encoding circuit search (MCS) that optimizes a cross-validation-based reward while respecting hardware constraints and circuit depth, leading to sample-efficient exploration. Across regression and classification tasks, MCS circuits consistently outperform literature reference circuits and genetic/random baselines, with QSVMs leveraging the projected quantum kernel for efficient evaluation. The findings demonstrate the practical impact of automated, data-driven circuit design for QML and point to future extensions to other QML models and symmetry-aware architectures.

Abstract

Quantum machine learning models use encoding circuits to map data into a quantum Hilbert space. While it is well known that the architecture of these circuits significantly influences core properties of the resulting model, they are often chosen heuristically. In this work, we present a novel approach using reinforcement learning techniques to generate problem-specific encoding circuits to improve the performance of quantum machine learning models. By specifically using a model-based reinforcement learning algorithm, we reduce the number of necessary circuit evaluations during the search, providing a sample-efficient framework. In contrast to previous search algorithms, our method uses a layered circuit structure that significantly reduces the search space. Additionally, our approach can account for multiple objectives such as solution quality, hardware restrictions and circuit depth. We benchmark our tailored circuits against various reference models, including models with problem-agnostic circuits and classical models. Our results highlight the effectiveness of problem-specific encoding circuits in enhancing QML model performance.

Figures (5)

• Figure 1: Automatic encoding circuit generation sketch. The MuZero algorithm observes the current circuit configuration, and transforms it into a hidden state representation $s_t^k$. By performing a Monte Carlo tree search for each time step $t$ the model obtains a policy $\Pi_t$ from which an action $a_t$ is sampled Schrittwieser_2020. The action corresponds to choosing a unitary operator (e.g., $H, CX,...$) from a set of operators $\mathcal{V}$ that is appended on the current encoding circuit in a layered fashion (the same gate for each qubit). Each circuit $U_t$ is used to encode data for a QML model. Based on the cross-validation score of this model, the MuZero agent receives a reward $r_t$ at each time step. The calculation of the immediate rewards, value function, prediction function and policy during the Monte Carlo tree search, and the replay buffer handling follows the concept described in Schrittwieser_2020, using fully-connected neural networks to approximate the functions.
• Figure 2: Example of a circuit with a layered structure with $q=4$ qubits that has been created after 5 actions of the RL agent. The operators are chosen from the action set described in Tab. \ref{['tab: action_set']}.
• Figure 3: Performance on the regression tasks. Results are shown for the California housing data (a) and the QFMNIST data (b). The first category marks the results of a quantum support vector regression (QSVR) based on encoding circuits built by the MCS (Sec. \ref{['sec:circuit_gen']}). The second category shows the results of a QSVR based on a variety of reference encoding circuits (Sec. \ref{['sec: reference_circs']}). The third and fourth columns mark the results of a QSVR based on random encoding circuits (Sec. \ref{['sec: random_circs']}). The fifth category shows the results of an encoding circuit built by a genetic algorithm (Sec. \ref{['sec: genetic_circs']}). The category CML shows the results of different classical ML models, as described in Sec. \ref{['sec: cml']}. The dashed lines mark the median performance of each category.
• Figure 4: Performance on the two-curves (diff) classification task. The columns follow the description in Fig. \ref{['fig: combined_results']}, with the QSVR models replaced by quantum support vector classification models, and dotted lines as median performance.
• Figure 5: Reference circuit architectures from the literature. (a) is taken from Hubreg_, (b) from kreplin2023reduction, (c) from Haug_2023, and (d) from peters2021machine. All graphics show one layer-block of the respective architectures.