Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Attention for Vision Transformers in High Energy Physics

Alessandro Tesi, Gopal Ramesh Dahale, Sergei Gleyzer, Kyoungchul Kong, Tom Magorsch, Konstantin T. Matchev, Katia Matcheva

TL;DR

The results indicate that embedding quantum orthogonal transformations within the attention mechanism can provide robust performance while offering promising scalability for machine learning challenges associated with the upcoming High Luminosity Large Hadron Collider.

Abstract

We present a novel hybrid quantum-classical vision transformer architecture incorporating quantum orthogonal neural networks (QONNs) to enhance performance and computational efficiency in high-energy physics applications. Building on advancements in quantum vision transformers, our approach addresses limitations of prior models by leveraging the inherent advantages of QONNs, including stability and efficient parameterization in high-dimensional spaces. We evaluate the proposed architecture using multi-detector jet images from CMS Open Data, focusing on the task of distinguishing quark-initiated from gluon-initiated jets. The results indicate that embedding quantum orthogonal transformations within the attention mechanism can provide robust performance while offering promising scalability for machine learning challenges associated with the upcoming High Luminosity Large Hadron Collider. This work highlights the potential of quantum-enhanced models to address the computational demands of next-generation particle physics experiments.

Quantum Attention for Vision Transformers in High Energy Physics

TL;DR

The results indicate that embedding quantum orthogonal transformations within the attention mechanism can provide robust performance while offering promising scalability for machine learning challenges associated with the upcoming High Luminosity Large Hadron Collider.

Abstract

We present a novel hybrid quantum-classical vision transformer architecture incorporating quantum orthogonal neural networks (QONNs) to enhance performance and computational efficiency in high-energy physics applications. Building on advancements in quantum vision transformers, our approach addresses limitations of prior models by leveraging the inherent advantages of QONNs, including stability and efficient parameterization in high-dimensional spaces. We evaluate the proposed architecture using multi-detector jet images from CMS Open Data, focusing on the task of distinguishing quark-initiated from gluon-initiated jets. The results indicate that embedding quantum orthogonal transformations within the attention mechanism can provide robust performance while offering promising scalability for machine learning challenges associated with the upcoming High Luminosity Large Hadron Collider. This work highlights the potential of quantum-enhanced models to address the computational demands of next-generation particle physics experiments.

Paper Structure

This paper contains 22 sections, 8 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Architectures and Circuits
  3. Vision Transformers
  4. Patch Extraction and Embedding
  5. Transformer Encoder
  6. Classification with Auxiliary Features
  7. Quantum Circuits
  8. RBS Gates
  9. Vector Loading Circuits
  10. Orthogonal Layer Circuit
  11. Attention Coefficient Circuit
  12. Experiments
  13. Data
  14. Data Description
  15. Auxiliary Features: Transverse Momentum ($p_T$) and Effective Mass ($m_0$)
  16. ...and 7 more sections

Figures (7)

  • Figure 1: Architecture of the Vision Transformers proposed.
  • Figure 2: Decomposition of the $RBS(\theta)$ gate
  • Figure 3: 4-Qubit Vector Loading Circuit with Reconfigurable Beam Splitter (RBS) Gates.
  • Figure 4: Pyramid Circuit with RBS Gates
  • Figure 5: Quantum circuit to compute $|x_i^T W x_j|^2$, a single attention coefficient, using data loaders for $x_i$ and $x_j$, and a quantum orthogonal layer for $W$.
  • ...and 2 more figures