Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QuCoWE Quantum Contrastive Word Embeddings with Variational Circuits for NearTerm Quantum Devices

Rabimba Karanjai, Hemanth Hegadehalli Madhavarao, Lei Xu, Weidong Shi

TL;DR

QuCoWE introduces a framework for learning quantum-native word embeddings using shallow, hardware-efficient PQCs trained with a contrastive skip-gram objective. By encoding words via data re-uploading and ring entanglement, and scoring similarity with a fidelity-based head and a logit-transformed fidelity head that aligns with PMI semantics, the method links quantum information measures to distributional semantics. An entanglement-budget regularizer based on single-qubit purities mitigates barren plateaus, and depolarizing/readout noise is analyzed with error-mitigation hooks. Experiments on Text8 and WikiText-2 show competitive intrinsic and extrinsic performance against 50–100D classical baselines while reducing learned parameters per token, suggesting near-term quantum devices can support viable semantic representations. The results highlight a practical, resource-aware path toward quantum NLP, with clear avenues for hardware deployment and extensions to contextualized embeddings.

Abstract

We present QuCoWE a framework that learns quantumnative word embeddings by training shallow hardwareefficient parameterized quantum circuits PQCs with a contrastive skipgram objective Words are encoded by datareuploading circuits with controlled ring entanglement similarity is computed via quantum state fidelity and passed through a logitfidelity head that aligns scores with the shiftedPMI scale of SGNSNoiseContrastive Estimation To maintain trainability we introduce an entanglementbudget regularizer based on singlequbit purity that mitigates barren plateaus On Text8 and WikiText2 QuCoWE attains competitive intrinsic WordSim353 SimLex999 and extrinsic SST2 TREC6 performance versus 50100d classical baselines while using fewer learned parameters per token All experiments are run in classical simulation we analyze depolarizingreadout noise and include errormitigation hooks zeronoise extrapolation randomized compiling to facilitate hardware deployment

QuCoWE Quantum Contrastive Word Embeddings with Variational Circuits for NearTerm Quantum Devices

TL;DR

QuCoWE introduces a framework for learning quantum-native word embeddings using shallow, hardware-efficient PQCs trained with a contrastive skip-gram objective. By encoding words via data re-uploading and ring entanglement, and scoring similarity with a fidelity-based head and a logit-transformed fidelity head that aligns with PMI semantics, the method links quantum information measures to distributional semantics. An entanglement-budget regularizer based on single-qubit purities mitigates barren plateaus, and depolarizing/readout noise is analyzed with error-mitigation hooks. Experiments on Text8 and WikiText-2 show competitive intrinsic and extrinsic performance against 50–100D classical baselines while reducing learned parameters per token, suggesting near-term quantum devices can support viable semantic representations. The results highlight a practical, resource-aware path toward quantum NLP, with clear avenues for hardware deployment and extensions to contextualized embeddings.

Abstract

We present QuCoWE a framework that learns quantumnative word embeddings by training shallow hardwareefficient parameterized quantum circuits PQCs with a contrastive skipgram objective Words are encoded by datareuploading circuits with controlled ring entanglement similarity is computed via quantum state fidelity and passed through a logitfidelity head that aligns scores with the shiftedPMI scale of SGNSNoiseContrastive Estimation To maintain trainability we introduce an entanglementbudget regularizer based on singlequbit purity that mitigates barren plateaus On Text8 and WikiText2 QuCoWE attains competitive intrinsic WordSim353 SimLex999 and extrinsic SST2 TREC6 performance versus 50100d classical baselines while using fewer learned parameters per token All experiments are run in classical simulation we analyze depolarizingreadout noise and include errormitigation hooks zeronoise extrapolation randomized compiling to facilitate hardware deployment

Paper Structure

This paper contains 37 sections, 3 theorems, 25 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivations and Challenges
  3. Related Work
  4. Classical Word Embeddings
  5. Quantum Natural Language Processing
  6. Parameterized Quantum Circuits
  7. Background
  8. Quantum States and Gradients
  9. Quantum Similarity
  10. Noise Models
  11. Method: QuCoWE Framework
  12. Architecture Overview
  13. Parameterized Quantum Circuit Design
  14. Similarity Scoring Heads
  15. Fidelity Head (F)
  16. ...and 22 more sections

Key Result

Theorem 1

For the fidelity-based loss with entanglement regularization, the gradient norm satisfies: where $B$ is the number of blocks and $\bar{E}$ is the average entanglement entropy across qubits.

Figures (3)

  • Figure 1: QuCoWE architecture: Words are encoded into parameterized quantum states via shallow PQCs. Similarity is computed through quantum fidelity and transformed via the logit-fidelity head for contrastive training.
  • Figure 2: Quantum circuit structure for QuCoWE with $B=2$ re-uploading blocks. Each block contains parameterized rotations and ring entanglement.
  • Figure 3: Sample efficiency on SST‑2

Theorems & Definitions (6)

  • Theorem 1: Gradient Scaling with Entanglement
  • proof
  • Theorem 2: LF Head Recovers PMI
  • proof
  • Theorem 3: Depolarizing Noise Effect
  • proof