Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processor

TL;DR

This work provides the first experimental robustness benchmark for a $20$‑qubit QNN on a superconducting quantum processor. It introduces Mask‑FGSM, a hardware‑aware, gradient‑sparsity–driven attack, and a fidelity‑based framework to quantify robustness bounds via $R_{\rm LB}$ and $R_{\rm UB}$, with an empirical gap of about $3\times 10^{-3}$ validating the bound tightness. Adversarial training markedly improves robustness by regularizing input gradients, and QNNs on hardware show stronger adversarial robustness than classical FNNs, a phenomenon attributed to gradient attenuation from intrinsic quantum noise. The study also demonstrates a practical, scalable pipeline for robustness benchmarking on NISQ devices and details the calibration, control, and readout mitigation required to realize reliable QNN experiments. Overall, this work lays a solid foundation for secure, reliable quantum machine learning on near‑term quantum hardware.

Abstract

Quantum machine learning (QML) models, like their classical counterparts, are vulnerable to adversarial attacks, hindering their secure deployment. Here, we report the first systematic experimental robustness benchmark for 20-qubit quantum neural network (QNN) classifiers executed on a superconducting processor. Our benchmarking framework features an efficient adversarial attack algorithm designed for QNNs, enabling quantitative characterization of adversarial robustness and robustness bounds. From our analysis, we verify that adversarial training reduces sensitivity to targeted perturbations by regularizing input gradients, significantly enhancing QNN's robustness. Additionally, our analysis reveals that QNNs exhibit superior adversarial robustness compared to classical neural networks, an advantage attributed to inherent quantum noise. Furthermore, the empirical upper bound extracted from our attack experiments shows a minimal deviation ($3 \times 10^{-3}$) from the theoretical lower bound, providing strong experimental confirmation of the attack's effectiveness and the tightness of fidelity-based robustness bounds. This work establishes a critical experimental framework for assessing and improving quantum adversarial robustness, paving the way for secure and reliable QML applications.

Figures (19)

• Figure 1: Experimental schematic for QNN robustness evaluation.$\textbf{a}$, Architecture of the QNN classifier, consisting of a state preparation circuit, an $l$-layer variational circuit, and pre-measurement basis transformation gates. $\textbf{b}$, Illustration of the robustness lower and upper bounds of the classifier. $\textbf{c}$, Adversarial robustness, quantified by the output sensitivity to input perturbations. $\textbf{d}$, Visualization of handwritten letters "Q" and "T" used in the image classification. $\textbf{e}$, Quantum synthetic dataset for LCEI, illustrating the application of an $R_x(\alpha)$ after the linear cluster state, with states labeled as "excited" or "non-excited" based on the rotation angle $\alpha$. $\textbf{f}$, The proposed masked adversarial attack is characterized by identifying the vulnerable subspace through input-gradient analysis and applying localized perturbations. $\textbf{g}$, Schematic of the superconducting quantum processor, showing $72$ qubits and $126$ couplers in a 2D lattice, and $20$ qubits selected for the experiment are highlighted in green.
• Figure 2: Experimental benchmarking of robustness bounds.$\textbf{a}$-$\textbf{d}$, Prediction probability $p$ and infidelity $D(\rho,\sigma)$ as functions of normalized perturbation strength $\hat{\epsilon}$. Panels $\textbf{a}$ and $\textbf{c}$ show robustness upper bound $R_{\rm{UB}}$ benchmarking experiments for two EMNIST samples corresponding to handwritten letters"Q" and "T", respectively. Panels $\textbf{b}$ and $\textbf{d}$ present analogous results for non-excited and excited cluster states in the LCEI task. Data points ($n=5$ independent experiments) are shown as mean $\pm$ standard deviation (SD). Solid lines represent fitting curves of $p_1$ and $D(\rho,\sigma)$, while the dashed purple/green lines mark the extracted values of $R_{\rm{UB}}$ at $\hat{\epsilon}^*$. $\textbf{e}$, $\textbf{f}$, Comparison of experimentally extracted upper bound $R_{\rm{UB}}$ (from $10$ randomly selected samples, $5$ per class) versus the theoretical lower bound $R_{\rm{LB}}$. Error bars indicate the root mean square error from fitting $D(\rho,\sigma)$. The averaged gap $\overline{\Delta}$ indicates the average gap between $R_{\rm{UB}}$ and $R_{\rm{LB}}$ across the $10$ samples, as defined under the infidelity metric.
• Figure 3: Adversary robustness benchmarking experiments.$\textbf{a}$, $\textbf{b}$, QNN sensitivity experiment for EMNIST (a) and LCEI (b). Purple and green indicate distinct classes. Datas ($n=5$ independent experiments) are presented in the form of mean values $\pm$ SD, with solid and dashed lines denoting linear fits for clean and adversarial trained models, respectively. $\textbf{c}$, Diagram of the local classification landscape, illustrating how adversarial training reorients input gradients to enhance robustness. $\textbf{d}$, $\textbf{e}$, Comparison of average input gradients for images "Q" ($\textbf{d}$) and "T" ($\textbf{e}$). The top panels show per-pixel gradients, while the bottom panels show flattened gradient vectors to facilitate observation. $\textbf{f}$, Comparison of average input gradients for LCEI states. The top panel shows the gradients of rotation angle for excited clusters, while the bottom for non-excited clusters. $\textbf{g}$, $\textbf{h}$, Correlation between the sensitivity $S$ and cosine similarity $\operatorname{cos}(\boldsymbol{\delta},\nabla_{\boldsymbol{x}} \mathcal{L})$ for EMNIST ($\textbf{g}$) and LCEI ($\textbf{h}$), with triangles and circles for different classes. The right panels show the sensitivity distribution under clean and adversarial training. $\textbf{i}$, Adversarial robustness scores of QNNs across different datasets under various training strategies, along with comparisons to FNN.
• Figure 4: Distribution of $\alpha$ for the synthetic quantum dataset in LCEI. Quantum data for the two classes are sampled from $\alpha \in [0,{3\pi}/{8}]$ (purple dashed lines) and $[{5\pi}/{8},\pi]$ (green dashed lines).
• Figure 5: Training and attack results.$\textbf{a}$, $\textbf{b}$, Clean training: loss, training accuracy, and test accuracy as functions of epoch. $\textbf{c}$, $\textbf{d}$, Adversarial training: loss, accuracy on legitimate samples, and accuracy on adversarial samples as functions of epoch. $\textbf{e}$, $\textbf{f}$, Accuracy of perturbed samples under Mask FGSM attacks, plotted as functions of normalized perturbation strength. Panels $\textbf{a}$, $\textbf{c}$ and $\textbf{e}$ correspond to EMNIST dataset, while $\textbf{b}$, $\textbf{d}$ and $\textbf{f}$ to LCEI.

Theorems & Definitions (2)

• Definition 1: Robustness lower bound
• Definition 2: Robustness upper bound