Benign Overfitting with Quantum Kernels

TL;DR

This paper tackles generalization in quantum kernel learning, where highly expressive quantum feature maps can cause kernel matrices to concentrate and hinder performance. It proposes Local-Global quantum kernels that combine a local, smooth kernel with a global, spiky component, and shows that a separable global encoding yields a tractable, tunable form $k_{LG}(x,z)=\tilde{\lambda}_L k(x,z)+\lambda_G k(x,z)^q$, enabling benign overfitting. Through angle- and Fourier-based experiments, the authors demonstrate that increasing the global component’s degree $q$ can interpolate training data while maintaining or improving test accuracy, supported by kernel eigenvalue analyses. The approach provides a practical pathway to harness quantum kernels on NISQ devices without sacrificing generalization, and offers a versatile design for constructing effective quantum feature maps. Overall, the Local-Global framework advances quantum kernel methods by explicitly balancing expressivity and generalization via spiky-smooth kernel design.

Abstract

Quantum kernels quantify similarity between data points by measuring the inner product between quantum states, computed through quantum circuit measurements. By embedding data into quantum systems, quantum kernel feature maps, that may be classically intractable to compute, could efficiently exploit high-dimensional Hilbert spaces to capture complex patterns. However, designing effective quantum feature maps remains a major challenge. Many quantum kernels, such as the fidelity kernel, suffer from exponential concentration, leading to near-identity kernel matrices that fail to capture meaningful data correlations and lead to overfitting and poor generalization. In this paper, we propose a novel strategy for constructing quantum kernels that achieve good generalization performance, drawing inspiration from benign overfitting in classical machine learning. Our approach introduces the concept of local-global quantum kernels, which combine two complementary components: a local quantum kernel based on measurements of small subsystems and a global quantum kernel derived from full-system measurements. Through numerical experiments, we demonstrate that local-global quantum kernels exhibit benign overfitting, supporting the effectiveness of our approach in enhancing quantum kernel methods.

Figures (7)

• Figure 1: Quantum fidelity kernel. When the input qubit register, here consisting of $t$ qubits, is initialized in $\ket{0^t}$, i.e., all the qubits in the register are initialized in $\ket{0}$, the circuit prepares the quantum state $U^\dagger(z)U(x)\ket{0^t}$. The result of measuring the observable $\ket{0^t}\bra{0^t}$ determines the kernel value, given by $k(x,z)= |\braket{\phi_x| \phi_z}|^2=|\braket{0^t | U^\dagger(z) U(x) | 0^t}|^2$.
• Figure 2: Local-global kernels for with $k_{LG}(x,z) =\cos^2(\frac{c(x-z)}{2}) + \rho\cos^{2^q}(\frac{c(x-z)}{2})$, where $c=\frac{3 \pi}{4}$, $\rho = 0.5$ and $q = 4,8,16$. As we will see in Section \ref{['sec:xp']}, the kernel $k(x,z) = \cos^2(\frac{c(x-z)}{2})$ is the fidelity quantum kernel obtained by angle quantum encoding canatar2022bandwidth.
• Figure 3: A quantum circuit for computing the local-global kernel $k_{LG}$ using \ref{['eq:sigma_tilde_xz']} and \ref{['eq:klg_c1']}. The measurement operator $O$ is defined as $O = \tilde{\lambda}_L O_L + \lambda_G O_G$, where $O_L = \ket{0^s}\bra{0^s} \otimes I_s^{\otimes q-1}$ and $O_G= (\ket{0^s}\bra{0^s})^{\otimes q}$.
• Figure 4: A quantum circuit for computing the local kernel $k_L$. The local-global quantum kernel $k_{LG}(x,z)$ is then obtained from $k_L$ through classical post-processing using \ref{['eq:eq']}.
• Figure 5: The kernel matrices of the quantum fidelity kernel with angle encoding are shown for different values of the dimension $d$, i.e., as the number of qubits increases. In angle encoding, data from $x\in\mathcal{U}([-1,1]^d)$ is mapped to the quantum state $\ket{\psi_x} = \bigotimes^d_{i=1} R_X(cx^{(i)})\ket{0}$, which leads to the quantum kernel $k_c(x,z)= \prod^d_{i=1} \cos^2(\frac{c(x^{(i)}-z^{(i)})}{2})$.

Theorems & Definitions (1)

• Definition 1: Local-Global Quantum Kernel