Quantum Mixed-State Self-Attention Network

TL;DR

The paper introduces Quantum Mixed-State Self-Attention Network (QMSAN), a hybrid quantum-classical framework that computes attention in the quantum domain using mixed-state representations and a swap-test circuit. It introduces a trainable quantum embedding to produce mixed-state queries/keys and pure values, along with a fixed quantum positional encoding scheme that leverages $R_x(\cdot)$ gates without extra qubits. Across MC, RP, and sentiment datasets, QMSAN variants outperform prior quantum self-attention models (QSANN) and competitive classical baselines, with positional encoding providing additional gains; the approach also demonstrates robustness to realistic quantum noise. These results suggest that retaining quantum information during similarity estimation (via $\alpha_{s,j}=\mathrm{tr}(\rho_{s,q}\sigma_{j,k})$) and efficient quantum encoding can enhance NLP tasks on near-term devices.

Abstract

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.

Figures (8)

• Figure 1: Quantum Mixed-State Self-Attention Network Framework.
• Figure 2: Comparison of Pure and Mixed States on Bloch Sphere.
• Figure 3: Circuits considered for comparing two-qubit interaction configurations.
• Figure 4: Quantum circuit implementing the SWAP test.
• Figure 5: Quantum Positional Encoding in Ring Configuration.