Quantum Complex-Valued Self-Attention Model

TL;DR

This paper addresses limitations of existing quantum self-attention by introducing Quantum Complex-Valued Self-Attention Model (QCSAM), which derives complex-valued attention weights from the complex inner product $\langle K|Q\rangle$ to capture amplitude and phase relationships. It extends Linear Combination of Unitaries to Complex LCUs (CLCUs) and enables quantum multi-head self-attention, embedding complex coefficients directly into circuit operations. The approach is validated on MNIST and Fashion-MNIST using 4-qubit implementations, achieving 100% and 99.2% test accuracies respectively, and demonstrates scalability to 3–8 qubits and 2–4 classes, with ablations showing advantages over real-valued weights. Overall, QCSAM advances quantum machine learning by enhancing the expressiveness and precision of quantum self-attention, aligning with the intrinsic complex nature of quantum mechanics and offering practical gains for quantum classifiers.

Abstract

Self-attention has revolutionized classical machine learning, yet existing quantum self-attention models underutilize quantum states' potential due to oversimplified or incomplete mechanisms. To address this limitation, we introduce the Quantum Complex-Valued Self-Attention Model (QCSAM), the first framework to leverage complex-valued similarities, which captures amplitude and phase relationships between quantum states more comprehensively. To achieve this, QCSAM extends the Linear Combination of Unitaries (LCUs) into the Complex LCUs (CLCUs) framework, enabling precise complex-valued weighting of quantum states and supporting quantum multi-head attention. Experiments on MNIST and Fashion-MNIST show that QCSAM outperforms recent quantum self-attention models, including QKSAN, QSAN, and GQHAN. With only 4 qubits, QCSAM achieves 100% and 99.2% test accuracies on MNIST and Fashion-MNIST, respectively. Furthermore, we evaluate scalability across 3-8 qubits and 2-4 class tasks, while ablation studies validate the advantages of complex-valued attention weights over real-valued alternatives. This work advances quantum machine learning by enhancing the expressiveness and precision of quantum self-attention in a way that aligns with the inherent complexity of quantum mechanics.

Figures (10)

• Figure 1: The framework of the proposed Quantum Complex-Valued Self-Attention Model (QCSAM).
• Figure 2: Enhanced Hadamard Test Circuit for Measuring Real and Imaginary Parts of Quantum Complex Self-Attention Weights.
• Figure 3: The architecture of 3 qubits circuit for encoding quantum attention weights.
• Figure 4: The architecture of CLCUs for implementing $A\ket{\psi} = \frac{1}{\mathcal{N}'}(\alpha_1U_{V1} + \alpha_2U_{V2} + \alpha_3U_{V3} + \alpha_4U_{V4})\ket{\psi}$
• Figure 5: Scalability of Our Models on MNIST with Varying Qubits, Classification Tasks, and Multi-Head Attention.

Theorems & Definitions (2)

• Definition 1: Quantum Complex Self-Attention Weight
• Definition 2: Quantum Complex Linear Combination of Unitaries, CLCUs