Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Embedding with Transformer for High-dimensional Data

Hao-Yuan Chen, Yen-Jui Chang, Shih-Wei Liao, Ching-Ray Chang

TL;DR

This work tackles high-dimensional binary classification by integrating Vision Transformer features with a trainable linear layer that parameterizes a quantum embedding feeding a quantum neural network. The hybrid pipeline employs a Pauli-feature-map-based quantum kernel and a Real Amplitude Ansatz, trained end-to-end with backpropagation on near-term devices. Empirical results on BirdCLEF-2021 show the transformer-based quantum embedding achieves higher F1 scores and far lower variability than CNN-based baselines, indicating both effectiveness and reliability. The study suggests scalability to larger quantum circuits and points to future exploration of quantum kernel methods for CV and NLP tasks in a hybrid quantum-classical framework.

Abstract

Quantum embedding with transformers is a novel and promising architecture for quantum machine learning to deliver exceptional capability on near-term devices or simulators. The research incorporated a vision transformer (ViT) to advance quantum significantly embedding ability and results for a single qubit classifier with around 3 percent in the median F1 score on the BirdCLEF-2021, a challenging high-dimensional dataset. The study showcases and analyzes empirical evidence that our transformer-based architecture is a highly versatile and practical approach to modern quantum machine learning problems.

Quantum Embedding with Transformer for High-dimensional Data

TL;DR

This work tackles high-dimensional binary classification by integrating Vision Transformer features with a trainable linear layer that parameterizes a quantum embedding feeding a quantum neural network. The hybrid pipeline employs a Pauli-feature-map-based quantum kernel and a Real Amplitude Ansatz, trained end-to-end with backpropagation on near-term devices. Empirical results on BirdCLEF-2021 show the transformer-based quantum embedding achieves higher F1 scores and far lower variability than CNN-based baselines, indicating both effectiveness and reliability. The study suggests scalability to larger quantum circuits and points to future exploration of quantum kernel methods for CV and NLP tasks in a hybrid quantum-classical framework.

Abstract

Quantum embedding with transformers is a novel and promising architecture for quantum machine learning to deliver exceptional capability on near-term devices or simulators. The research incorporated a vision transformer (ViT) to advance quantum significantly embedding ability and results for a single qubit classifier with around 3 percent in the median F1 score on the BirdCLEF-2021, a challenging high-dimensional dataset. The study showcases and analyzes empirical evidence that our transformer-based architecture is a highly versatile and practical approach to modern quantum machine learning problems.
Paper Structure (15 sections, 5 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 15 sections, 5 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Model Architecture
  4. Vision Transformer
  5. Trainable Quantum Embedding with Classical Linear Transformation
  6. Mathematical Model
  7. Quantum Feature Map and Ansatz
  8. Results
  9. Effectiveness on Binary Classification
  10. Performance Reliability
  11. Discussion
  12. Extension to Larger Circuits
  13. Technical Challenges
  14. Outlooks
  15. Conclusion

Figures (4)

  • Figure 1: Model architecture of the research on quantum embedding with transformers for high-dimensional visual datasets. The orange part stands for the classical state of data, and the blue part means a quantum state of information. The input data feeds from the left to the right to classify the input information.
  • Figure 2: Quantum neural network's architecture. Once the data processed by the transformer, the linear representation layer will transform the data and embed into the quantum circuit forming quantum state of information.
  • Figure 3: Comparative results for the BirdCLEF-2021 dataset over various embedding methods, including classical CNN method (at the far left), CNN-based quantum embedding (at the middle), the architecture proposed in this research, which is transformer-based quantum embedding (at the far right)
  • Figure 4: Comparative results for the standard deviation of F1 score for BirdCLEF-2021 dataset over various embedding methods or training method, including classical CNN method (at the top), CNN-based quantum embedding (at the middle), the architecture proposed in this research, which is transformer-based quantum embedding (at the button)