Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Vision Transformer and Quantum Convolutional Neural Network for Image Classification

Mingzhu Wang, Yun Shang

TL;DR

This work presents ViT-QCNN-FT, a hybrid quantum-classical framework that fuses a fine-tuned Vision Transformer with a quantum convolutional neural network to compress high-dimensional image data for NISQ devices. Through systematic analysis of quantum encoding, QCNN architecture, and noise, it demonstrates that amplitude encoding with uniformly entangled QCNN ansatzes yields high accuracy (up to $99.77\%$ on CIFAR-10) and that substituting QCNNs with classical counterparts degrades performance by around $29.36\%$, evidencing quantum advantage. The study also reveals noise can act as a regularizer in certain regimes and provides design guidelines for entanglement distribution, pooling, and trainability in quantum feature extractors. Collectively, these results establish a principled pathway for co-designing classical and quantum components toward practical QML on high-dimensional tasks.

Abstract

Quantum machine learning (QML) holds promise for computational advantage, yet progress on real-world tasks is hindered by classical preprocessing and noisy devices. We introduce ViT-QCNN-FT, a hybrid framework that integrates a fine-tuned Vision Transformer with a quantum convolutional neural network (QCNN) to compress high-dimensional images into features suited for noisy intermediate-scale quantum (NISQ) devices. By systematically probing entanglement, we show that ansatzes with uniformly distributed entanglement entropy consistently deliver superior non-local feature fusion and state-of-the-art accuracy (99.77% on CIFAR-10). Surprisingly, quantum noise emerges as a double-edged factor: in some cases, it enhances accuracy (+2.71% under amplitude damping). Strikingly, substituting the QCNN with classical counterparts of equal parameter count leads to a dramatic 29.36% drop, providing unambiguous evidence of quantum advantage. Our study establishes a principled pathway for co-designing classical and quantum architectures, pointing toward practical QML capable of tackling complex, high-dimensional learning tasks.

Hybrid Vision Transformer and Quantum Convolutional Neural Network for Image Classification

TL;DR

This work presents ViT-QCNN-FT, a hybrid quantum-classical framework that fuses a fine-tuned Vision Transformer with a quantum convolutional neural network to compress high-dimensional image data for NISQ devices. Through systematic analysis of quantum encoding, QCNN architecture, and noise, it demonstrates that amplitude encoding with uniformly entangled QCNN ansatzes yields high accuracy (up to on CIFAR-10) and that substituting QCNNs with classical counterparts degrades performance by around , evidencing quantum advantage. The study also reveals noise can act as a regularizer in certain regimes and provides design guidelines for entanglement distribution, pooling, and trainability in quantum feature extractors. Collectively, these results establish a principled pathway for co-designing classical and quantum components toward practical QML on high-dimensional tasks.

Abstract

Quantum machine learning (QML) holds promise for computational advantage, yet progress on real-world tasks is hindered by classical preprocessing and noisy devices. We introduce ViT-QCNN-FT, a hybrid framework that integrates a fine-tuned Vision Transformer with a quantum convolutional neural network (QCNN) to compress high-dimensional images into features suited for noisy intermediate-scale quantum (NISQ) devices. By systematically probing entanglement, we show that ansatzes with uniformly distributed entanglement entropy consistently deliver superior non-local feature fusion and state-of-the-art accuracy (99.77% on CIFAR-10). Surprisingly, quantum noise emerges as a double-edged factor: in some cases, it enhances accuracy (+2.71% under amplitude damping). Strikingly, substituting the QCNN with classical counterparts of equal parameter count leads to a dramatic 29.36% drop, providing unambiguous evidence of quantum advantage. Our study establishes a principled pathway for co-designing classical and quantum architectures, pointing toward practical QML capable of tackling complex, high-dimensional learning tasks.
Paper Structure (18 sections, 22 equations, 5 figures, 3 tables)

This paper contains 18 sections, 22 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Results
  3. Encoding comparison
  4. Noise robustness
  5. Feature extractor ablation
  6. Quantum efficiency
  7. Methods
  8. Feature extraction
  9. Quantum state encoding
  10. Amplitude Encoding
  11. Angle encoding
  12. Dense angle encoding
  13. Quantum convolution neural network
  14. Convolution ansatzes and pooling ansatz
  15. Quantum convolution neural network ansatzes
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Overall model diagram of ViT-QCNN-FT and parameterized quantum circuits utilized in the convolutional and pooling layers. The pre-trained ViT is fine-tuned to act as a feature extractor. The extracted features are then encoded into quantum states (green block), followed by a QCNN performing the classification task. The QCNN consists of two primary components: convolutional filters (blue blocks) and pooling operations (red circles). $R_{\sigma}(\theta)$ represents a rotation gate around the $\sigma$-axis of the Bloch sphere by an angle $\theta$, while $H$ denotes the Hadamard gate. $U3(\theta,\phi,\lambda)$ is a general single-qubit gate, which can be expressed as $U3(\theta,\phi,\lambda)=R_z(\phi)R_z(-\pi/2)R_z(\theta)R_z(\pi/2)R_z(\lambda)$.
  • Figure 2: a, The average accuracies of 9 ansatzes under three quantum state encoding methods (10 qubits). b, Comparison of pooling ansatzes and no-pooling ansatzes under three quantum state encoding methods (10 qubits). c, The results of 18 ansatzes under the angle encoding (10 qubits). d, The results of 18 ansatzes under amplitude encoding (8 qubits).
  • Figure 3: a, The influence of noise intensity on the results under different types of quantum noise. b, The effects of pooling and no-pooling under different noise types and intensities. c, The average accuracy of ansaztes under different noise intensities. d, The results of 18 ansatzes under the depolarization noise intensity of 0.05.
  • Figure 4: a, This work is compared with eight other models. The horizontal axis represents the standard deviation, and the vertical axis represents the accuracy. Each point is derived from the average result of the 18 ansatzes under this model. b, The sampling entanglement entropy distribution of convolution 7. c, The sampling entanglement entropy distribution of convolution 8. d, The sampling entanglement entropy distribution of convolution 9.
  • Figure 5: a, Comparison between Ansatz 3 no-pooling(A3’) and other classic CNNs. The horizontal axis represents the standard deviation, and the vertical axis represents the accuracy. b, T-SNE of CNN1. c, T-SNE of CNN2. d, T-SNE after the first layer of convolution and pooling in A3'. e, T-SNE after the second layer of convolution and pooling in A3'. f, T-SNE after the third layer of convolution and pooling in A3'.